Global De-Duplication

[charles@opentelcom.com]

Hi Everyone,

Does anyone know a way to do a global deduplication?

I have about 4000 Accounts, and each seems to have been imported 5 times.

Going through and selecting each set of 5 accounts and manually merging them is just not an option, does anyone have any suggestions?

[chrisky]

Hi Everyone,

Does anyone know a way to do a global deduplication?

I have about 4000 Accounts, and each seems to have been imported 5 times.

Going through and selecting each set of 5 accounts and manually merging them is just not an option, does anyone have any suggestions?

Yeah...

there is a handful of options in which you have..

The first would be a good backup routine..

Since I do not personally know of a utility or program in which does such (though I'm sure that such type of app does exist), I'd just code such utility in PHP..

Since, obviously both those options are unknown and/or not possible in your case (use of db utility app to remove duplicates and/or programming such a utility in PHP), I realy don't have a solution for you.

Maybe another forum member might be aware of or know of, an utility program (PHP based preferrably) in which is capable of identifying and removing duplicate db records?

And, in light of such difficulties, I would encourage you to put more importance, focus, and attention, on performing DB backups..

And you might keep an eye on the BackupRestore thread, as it will be within days, in which I will be releasing that..

At this point, just touch up's are being done to the first portion to be publically released, and work is being done on packaging it up as a SugarCRM Module in which is module installable via Sugar.

Anyways, good luck with it, and I hope that someone might be able to suggest an app or utility in which will remove those duplicate records for you.

[lvangool]

Since you have SugarCRM, I assume you have phpMyAdmin for database work. If not, get it...

Now go to the right database and input this SQL query:

SELECT CONCAT(last_name, first_name) AS fullname, * FROM accounts GROUP BY fullname

Then, in the results window, click Export...make sure to export to CSV. Open that CSV file in MS Excel or OOCalc and remove the fullname column. Then, using phpmyadmin again, you can clear out the accounts table (make sure to backup..) and insert the csv file.