Allowed memory size of 262144000 bytes exhausted

[richmarotta1]

I have 1 million targets and am trying to add all them to a target list via a select. It stopped after adding about 12000 records. I then upped the max memory allocation in /etc/php.ini to 524288000. The app failed after adding about 30000 records to target list.

What can I change in either my Mysql, Sugar, or php config that will allow the app to continue until all the targets are added?

Sugar version CE 6.1
Mysql Version5.0.77-log
PHP Version 5.2.10

Thanks,
Rich

[kuske]

1.000.000 Tagets?
Are you sure that anyone on the world is really interested in your product?????

solution:
use small steps, add all targets starting with aa, then ab, then ac, and so on.

[richmarotta1]

That would be very painful way to do this.
How do large organizations with millions of records use sugar if it has this type of limitation?

[kuske]

Sugar is no mass mailer - it is a customer relationship system.
That means that it helps you to communicate with your customers.
A mass mailer is uses for one way communication.
Look for mass mailing tools, not for crm systems.

[richmarotta1]

I was considering that but there were a couple issues:
1. the cost of using 1M contacts on those services is ~1000/month.
2. I have integrated the Sugar trackers into the email. It seems that the emails need to be sent by Sugar for those to work. There may be a workaround for that. Not sure.
3. My tests worked great. If it is just a matter of bulk the I'll figure out how to get round the memory limit.

Do you have any suggestions regarding the memory limit problem?

Rich

[kuske]

There are open source mass mailers too.

Your memory issue is very easy to solve.
For 30.000 Targets you need 0,5 GByte RAM
For 300.000 Targets ... 5 GByte
For 1.000.000 Targets ... about 15 GByte.

I'm mean, I know it.

The real problem is, that sugar handles each record with full workflow and that needs memory. In a mass mailer only the master data of the record is used to send the email. Best practice would be to connect the mass emailer responses with the sugar bounce box to generate leads in the crm system. Such solutions exist, I know.

[richmarotta1]

I did that math but was hoping for a more realistic solution. We are new to Sugar. Limitations like this makes it seem like Sugar is not for large organizations.

[kuske]

Sugar is not the best practice for large mass emails.
It does not have problems with lage organisations.
You can work on it with hundreds of users without problems.
Only the mass emailing of millions of targets is not a job for sugar.

[ajlisowski]

If something like this was really needed, you would likely need to customize sugar to run only a section of the targets at a time. Have a javascript script that makes ajax calls to the sugar script which would then send out 1000 of the mails. Then it would send out the next 1000 until all your targets have been sent.

[kuske]

If your server is really a fast one you can send perhaps 100 Emails per minute!
That means 6000 per hour, 144.000 in a day, 1.000.000 in 7 days, sending permanently...
This numbers shows that you really need a more dedicated mailing system.