SOAP Performance Issue Loading Accounts

[artisticlight]

I have been delving into Sugar for the past couple of weeks and have worked on moving account data from an existing order processing system into the Sugar Accounts module. My latest effort was using Sugar's exposed SOAP interface (set_entry) to load 4,000+ records. I am having terrible performance problems.

It takes approximately 3 minutes for my php script to run and load 50 accounts. If I try to load 100 accounts at a time then I get a timeout error indicating I've exceeded the 300 second maximum. I want to load large volumes of data on a regular basis into accounts and other existing and new modules I plan to create. Is SOAP really viable for this, or is it only good for loading a handful of records at at time?

I wrote php scripts that load the mysql accounts table directly and I used Jitterbit to load data. They both handled the same 4,000+ records within a matter of seconds.

Is SOAP that inefficient, or am I missing something?

I wanted to use the SOAP approach to make sure the data gets loaded properly, including necessary relationships. I don't know that there are published specs regarding those relationship, and I suspect the specs will change over time. Hence, I thought SOAP would ensure that the data got loaded in a manner that Sugar would endorse, and not be as prone to being broke by Sugar upgrades and changes to their data structures.

Any advice would be appreciated. Use SOAP and live with pathetic performance? Ways to increase SOAP performance? Add data by directly accessing the tables and forget SOAP?

Thanks in advance for your help.

Andrew Anderson

[eggsurplus]

SOAP is never a good method for converting data (except for the idea of creating items correctly). If you must, then bump up the timeout to a day or so just to be safe. I'd highly recommend doing a database to database conversion like you've done before. If you have a Sugar db set up with sample data you can see how it should be set up. It may take some guessing and checking but most conversions do.