Special French accent characters erasure

[SimonLessard]

Hello,

I wasn't sure on what forum to post this since it's specific to certain languages (only French?) , but since it happen even without any language pack installed I'm posting it here.

Sugar version: 4.2.0d
OS: Windows XP Professional Service Pack 2
Installation type: rPath Linux VMware Image
Language pack version: fr_FR LangPack SYNOLIA 4.2.1
VM player version: 1.0.1-19317
Keyboard setting: French (Canada)
Browser: Tested with Firefox and IE

Description:
It seems that some fields erase accent characters. The fields in problem are fields allowing free text, the list of values editor works fine. For the fields in problem, text can be entered normally, but when it's submitted, the parsing simply stop as soon as a character with an accent is reached. e.g. if you enter something like "hello! é test1 è test2 ê test3", the final text will be "hello! ".

Note: The error happens while using English language pack as well.

Reproduction case:
Install VM client
Download
Start the VM
Go to My Account
Click Edit
Change the "Notes" field's value to : "hello! é test1 è test2 ê test3" or any other string with an accent
Click Save
Check the result

Anyone ever heard about that issue and/or knows a fix for it by any chance?


Thanks!

Simon Lessard

[matthew]

Since these are multi-byte chars - they're not supported. The problem can also exist at the database layer or the PHP layer.

The good news is though, that our 4.5 release will focus heavily on internationalization and utf8 formats.

CHeers,

[SimonLessard]

Hello,

Thank for the reply, we'll take notice of the issue and not use accents for now.


Regards,

Simon Lessard

[SimonLessard]

Hello again,

Here's a little follow-up on that issue. After checking, none of the accent characters are multibyte for ISO-8859-1 character encoding. Furthermore, version 4.01 as well as 4.5 demos works fine with those characters, so my guess is that there's a problem in the VM configuration. I might install Windows version with a separated MySWL BD and PHP server to see if the issue still show or not.


Regards,

Simon Lessard

[Mikeqc]

I have the same problem over here...
It appears to be only with the VM I have downloaded (v4.2.0 at that time).
I don't know if it is fixed in the 4.2.1a currently available.

Somebody have an idea why? I am not very good with Linux, PHP and MySQL...
I don't know what config to look at...

Thanks,

Mike

[Mikeqc]

I drill down the problem.
It think it is on MySQL.
I tried doing some INSERT or UPDATE statements with special characters and it's not working...

Now, I'm trying to figure out why...

[nathanbthomas]

I am fascinated to hear the final explanation on this one. If it is something that needs to be fixed in the vmware image I'll get it respun immediately.

Is there a simple test case for this I can add to the release process?

Nathan

[Mikeqc]

Sure, currently I'm working on trying to make the database working with Sugar.
So, If you create a note and you put a string like "Vois-tu là bas?" in your description, the insert in the database will result in "Vois-tu l" and the rest will be truncated.

In the database, the insert in the "notes" table will pass but will have a warning 1265 ("Data truncated for column...").

So, I'm trying to find the correct settings for this database but I think I'll need some help... :-s

Mike

[Mikeqc]

Ok, I finally fix that on my server...
Here is what I've done:
in /etc/init.d/mysqld add the 2 following parameters in the start parameters
--character-set-server=latin1 --skip-character-set-client-handshake

After that, you may have to do on your database:
alter database sugarcrm character set latin1;

But for this one I'm not sure since I did it previously and it may have no impact.
So, try the mysqld first then if it's still not working, try to change the character set on the database level...

Hourray!

Mike

[nathanbthomas]

Alright, to keep things simply on my end I'm just going to add the following the /etc/my.cnf

character_set_server = latin1

I'll probably wait until 4.5 GA's to update the image.

Nathan