Error processing incoming emails (4.5b)

**ayavilevich** · 2006-09-15, 12:42 AM

When performing the "Bounced Campaign Emails" "Scheduler" task, some emails were processed correctly, but at some moment an

error appeared in the log:

09/14/06 16:35:26,443 [560] FATAL SugarCRM - ***ERROR: InboundEmail::handleCharsetTranslation() called without a $charset!
09/14/06 16:35:26,468 [560] FATAL SugarCRM - ***STACKTRACE: Array
XXXXstuffXXXX
09/14/06 16:35:27,703 [560] FATAL SugarCRM - ***ERROR: InboundEmail::handleCharsetTranslation() called without a $charset!
09/14/06 16:35:27,726 [560] FATAL SugarCRM - ***STACKTRACE: Array XXXXstuffXXXX
09/14/06 16:35:27,750 [560] FATAL SugarCRM - MySQL error 1406: Data too long for column 'description' at row 1
09/14/06 16:37:03,171 [2812] FATAL SugarCRM - ***ERROR: InboundEmail::handleCharsetTranslation() called without a $charset!
09/14/06 16:37:03,194 [2812] FATAL SugarCRM - ***STACKTRACE: Array XXXXstuffXXXX
09/14/06 16:37:03,213 [2812] FATAL SugarCRM - MySQL error 1406: Data too long for column 'description' at row 1

From that moment the "Scheduler" is stuck "In Progress".
We are running 4.5b .
The mail that is causing trouble seems to come from hotmail.
Any ideas of how to fix this?

Regards,
Arik.

(Win2003 server, IIS6, php5)

**ruchida** · 2006-09-15, 12:58 AM

Do you know what the charset of the incoming emails is?
It may be related to this issue.

**ayavilevich** · 2006-09-15, 10:35 AM

Here are the headers of the problematic email

Code:

Return-Path: <userx@hotmail.com> Thu Sep 14 14:05:23 2006
Received: from bay0-omc3-s40.bay0.hotmail.com [65.54.246.240] by skywalker.myinternetwebhost.com with SMTP;
   Thu, 14 Sep 2006 14:05:23 -0700
Received: from hotmail.com ([65.54.162.26]) by bay0-omc3-s40.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.1830);
	 Thu, 14 Sep 2006 14:05:22 -0700
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Thu, 14 Sep 2006 14:05:22 -0700
Message-ID: <BAY108-F162C9209317FC353914E0AC4290@phx.gbl>
Received: from 65.54.162.200 by by108fd.bay108.hotmail.msn.com with HTTP;
	Thu, 14 Sep 2006 21:05:21 GMT
X-Originating-IP: [89.1.21.68]
X-Originating-Email: [userx@hotmail.com]
X-Sender: userx@hotmail.com
In-Reply-To: <ff331c087d00786673e527bdb272b675@crm.domain.com>
From: "Ohad L" <userx@hotmail.com>
To: crm@domain.com
Bcc: 
Subject: RE: domain Beta Invitation
Date: Thu, 14 Sep 2006 21:05:21 +0000
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
X-OriginalArrivalTime: 14 Sep 2006 21:05:22.0307 (UTC) FILETIME=[7DE1ED30:01C6D841]
Return-Path: userx@hotmail.com

Content type has no charset attribute. So when
$msgPart = $this->handleCharsetTranslation($text, $decodedHeader['Content-Type']['charset']);
Tries to run it fires an error at

Code:

		if(empty($charset)) {
			$GLOBALS['log']->fatal("***ERROR: InboundEmail::handleCharsetTranslation() called without a \$charset!");
			$GLOBALS['log']->fatal("***STACKTRACE: ".print_r(debug_backtrace(), true));
			return $text;
		}

But as I see it, it shouldn't fail, right? Just show the error and fall back to default...

Which means that the part that fails is:
09/14/06 16:37:03,213 [2812] FATAL SugarCRM - MySQL error 1406: Data too long for column 'description' at row 1

How would I go about debugging this MySQL error? 'description' is a long text, how something small such an email is too big for it??

Thanks,
Arik.

**ayavilevich** · 2006-09-15, 11:48 AM

Ok, digged deeper.

Looks like the mising charset and the mysql error are related.
This mysql bug (win only) gives some info http://bugs.mysql.com/bug.php?id=18908 about the issue.

mysql is expecting text in utf8, but the incomming email is not in utf8 and sugar skips charcode conversion when there is no charset header in the email.
So the result is trying to insert ascii text into mysql and it fails.

I made a home-made patch to foce a default charset and it worked.
Changed InboundEmail.php at line 807 to say this:

Code:

			// AY
			if( !isset($decodedHeader['Content-Type']['charset']) || empty($decodedHeader['Content-Type']['charset']) )
				$msgPart = $this->handleCharsetTranslation($text, 'latin1');
			else
			// ~AY			
			$msgPart = $this->handleCharsetTranslation($text, $decodedHeader['Content-Type']['charset']);

Works for now. What do you think?

Arik.