Cannot Import Norwegian Characters

[magicbobeto]

Hi,

os: linux 2.6.24.3-grsec-sg1
sugar: 5.0.0.d
php: 5.2.5
webserver: 1.3.37 (Unix)
mysql: 5.0.45-community-log

I am having problems importing CSV files of Account and Contact records that contain Norwegian characters.

If I create a new account or contact directly in sugar I can enter Norwegian characters without problems.

However if I run an Import from Sugar, all the Norwegian characters are removed !! ( E.g 'Ålgård' becomes 'lgrd' ).

I have ensured that all the collations that i can set in MySQL are 'utf8_general_ci'. Below is a list of the MySQL Collations variables as shown through phpMyAdmin. Some of these still show latin1_swedish_ci', i believe these are server settings that i don't have rights to change.

Database Collation
information_schema utf8_general_ci
act_sugarprod utf8_general_ci
act_sugarprod2 utf8_unicode_ci
act_sugarskel utf8_unicode_ci
Total: 4 latin1_swedish_ci

character set client utf8
(Global value) latin1
character set connection utf8
(Global value) latin1
character set database latin1
character set filesystem binary
character set results utf8
(Global value) latin1
character set server latin1
character set system utf8
character sets dir /usr/share/mysql/charsets/
collation connection utf8_unicode_ci
(Global value) latin1_swedish_ci
collation database latin1_swedish_ci
collation server latin1_swedish_ci

It looks like the problem must be to do with the actual Import process from Sugar.

It's really strange problem that is causing me a lot of headaches, as i really need those Norse characters in there!! If anyone has any ideas on how to fix it please let me know!!!

Thanks,

Magic Bob

[roblaus]

Hi,

I just tested it and in 5e it works fine (and as far as I know it did in all previous versions).

Did you make sure that your import file is utf8? It sounds stupid but the best test is to import the csv into e.g. OO calc and check with which character set you Norwegian characters look right. If it's not utf8 you know where the problem is.
BTW. the database doesn't really care about the character set, it's the sort (collation) that must be set up correctly (a before ä and similar).

rgds
rl

[magicbobeto]

Brilliant Robert !!

That was the problem, i'd exported the CSV from Excel, but then I had to do a bit of editing in TextEdit, and when i saved that file it was saving it in Western encoding. Saving it as Unicode sorted all my problems.

You da man!!!!!

Thanks a lot!

Magic Bob