Bounce handling

[ysh]

Hi

I have read through a small sample (~20) of the posts concerning
the handling of bounced emails.

My client's current setup works fine. The campaign results bounce
email subpanel clearly displays bounced emails.

I do however have a few gripes about the way bounced emails are handled:

The quantity of NDRs received in the inbox of the email account
used for the campaign far out-numbers the registered bounced
tally displayed on the campaign subpanel :

- Out of 893 emails received that were related to this campaign,
in the account's inbox, 823 had the following subject "Mail delivery
failed: returning message to sender"

- Meanwhile the campaign bounced emails subpanel only noted
a meagre 131 bounced emails, and a single bounced email due
to an invalid address.

Furthermore, the "invalid email" tick box wasn't updated for the
contact whose address was determined as invalid.

Questions :

- Why is it that those 823 bounced emails don't appear in the tally
on the campaign subpanel ?

- If these 823 NDRs had been properly taken into account, would
they have appeared as bounced due to invalid addresses ?

- Are contacts supposed to get updated automatically when a
campaign detects invalid email addresses ?

Thanks for reading
yann

[ysh]

If a recipient clicks on the RemoveMe link, shouldn't this be reflected on his contact record ?

[martyp]

Sugar Version 6.4.0 (Build 7276) Corporate

My campaign has been working fine, with the bounce emails going into the correct email box. The only thing that doesn't appear to be working is the invalid email marking as part of the campaign process. There is a snippet of code in the modules/Campaigns/ProcessBouncedEmails.php file:

//todo mark the email address as invalid. search for prospects/leads/contact associated
//with this email address and set the invalid_email flag... also make email available.
$GLOBALS['log']->info("Warning: Invalid identifier for campaign log $identifier");
return FALSE;

Does anyone know if this process has been completed/fixed, what is required to complete this code, if in fact this is the problem with the invalid email linkage? Thanks in advance for any feedback you may have,.

[matherren]

This code in ProcessBouncedEmail.php work for me;

Line: 84

     //do we have the phrase permanent error in the email body.
    //  #REMOVE# if (preg_match('/permanent[ ]*error/',$email_description))
    if (preg_match('/permanent[ ]*error|permanent[ ]*failure|recipient[ ]*failed/',$email_description))
        $bounce->activity_type='invalid email';
    else
        $bounce->activity_type='send error';

    // #REMOVE# $return_id=$bounce->save();
    $bounce->save();

    // #REMOVE# return $return_id;
    return $bounce; 
  

Line: 188

                         // #REMOVE# $return_id = createBouncedCampaignLogEntry($row, $email, $email_description);
                        // #ADDED#
                        $bounce = createBouncedCampaignLogEntry($row, $email, $email_description);
                        if ($bounce->activity_type == 'invalid email')
                        {
                            $target_id   = $bounce->target_id;
                            $target_type = $bounce->target_type;
                            $GLOBALS['db']->query("UPDATE email_addresses
                                                          JOIN email_addr_bean_rel ON email_addresses.id=email_addr_bean_rel.email_address_id
                                                                                  AND email_addr_bean_rel.deleted=0
                                                                                  AND email_addr_bean_rel.primary_address=1
                                                                                  AND email_addr_bean_rel.bean_id='{$target_id}'
                                                                                  AND email_addr_bean_rel.bean_module='{$target_type}'
                                                           SET email_addresses.invalid_email=1
                                                         WHERE invalid_email=0");
                        };
                        // #ADDED# 
  

[martyp]

This code in ProcessBouncedEmail.php work for me;

Line: 84

     //do we have the phrase permanent error in the email body.
    //  #REMOVE# if (preg_match('/permanent[ ]*error/',$email_description))
    if (preg_match('/permanent[ ]*error|permanent[ ]*failure/',$email_description))
        $bounce->activity_type='invalid email';
    else
        $bounce->activity_type='send error';
 
    // #REMOVE# $return_id=$bounce->save();
    $bounce->save();
 
    // #REMOVE# return $return_id;
    return $bounce; 
  

Line: 188

                         // #REMOVE# $return_id = createBouncedCampaignLogEntry($row, $email, $email_description);
                        // #ADDED#
                        $bounce = createBouncedCampaignLogEntry($row, $email, $email_description);
                        if ($bounce->activity_type == 'invalid email')
                        {
                            $target_id   = $bounce->target_id;
                            $target_type = $bounce->target_type;
                            $GLOBALS['db']->query("UPDATE email_addresses
                                                          JOIN email_addr_bean_rel ON email_addresses.id=email_addr_bean_rel.email_address_id
                                                                                  AND email_addr_bean_rel.deleted=0
                                                                                  AND email_addr_bean_rel.primary_address=1
                                                                                  AND email_addr_bean_rel.bean_id='{$target_id}'
                                                                                  AND email_addr_bean_rel.bean_module='{$target_type}'
                                                           SET email_addresses.invalid_email=1
                                                         WHERE invalid_email=0");
                        };
                        // #ADDED# 
  

Hey Matherren

Thank you VERY much for the reply and most importantly the code. Probably a pretty 'dumb' question but it appears that this line

if (preg_match('/permanent[ ]*error|permanent[ ]*failure/',$email_description))

means that ONLY emails with the text 'permanent error' or 'permanent failure' in the email description will have their email addresses set to invalid? I have had a look through some of my 900 odd emails that are waiting for bounce processing and quite a few of them don't have this line. They have variations e.g.

Action: failed
Status: 5.1.1
Diagnostic-Code: smtp;550 5.1.1 RESOLVER.ADR.RecipNotFound; not found

Or

The following addresses had permanent fatal errors

and possibly other variants. I haven't looked through all 900 yet ;-] Would this replacement cover the variants I've listed?

if (preg_match('/permanent[ ]*error|permanent[ ]*failure|permanent[ ]*fatal|RecipNotFound[ ]*not found/',$email_description))

Thanks for your assistance and your time and patience

[matherren]

Hey Matherren

Thank you VERY much for the reply and most importantly the code. Probably a pretty 'dumb' question but it appears that this line

if (preg_match('/permanent[ ]*error|permanent[ ]*failure/',$email_description))

means that ONLY emails with the text 'permanent error' or 'permanent failure' in the email description will have their email addresses set to invalid?

Yes

I have had a look through some of my 900 odd emails that are waiting for bounce processing and quite a few of them don't have this line. They have variations e.g.

Action: failed
Status: 5.1.1
Diagnostic-Code: smtp;550 5.1.1 RESOLVER.ADR.RecipNotFound; not found

Or

The following addresses had permanent fatal errors

and possibly other variants. I haven't looked through all 900 yet ;-] Would this replacement cover the variants I've listed?

if (preg_match('/permanent[ ]*error|permanent[ ]*failure|permanent[ ]*fatal|RecipNotFound[ ]*not found/',$email_description))

Thanks for your assistance and your time and patience

rule "permanent[ ]*error" match the line too.

The following addresses had permanent fatal errors

permanent[ ]*fatal isn't necesary

Line: 188

                     // #REMOVE# $return_id = createBouncedCampaignLogEntry($row, $email, $email_description);
                    // #ADDED#
                        $bounce = createBouncedCampaignLogEntry($row, $email, $email_description);
                        if ($bounce->activity_type == 'invalid email')
                        {
                            $target_id   = $bounce->target_id;
                            $target_type = $bounce->target_type;
                            $GLOBALS['db']->query("UPDATE email_addresses
                                                   LEFT JOIN email_addr_bean_rel ON email_addresses.id=email_addr_bean_rel.email_address_id
                                                   SET email_addresses.invalid_email=1, email_addresses.date_modified=NOW()
                                                   WHERE email_addresses.invalid_email=0
                                                     AND email_addr_bean_rel.deleted=0
                                                     AND email_addr_bean_rel.primary_address=1
                                                     AND email_addr_bean_rel.bean_id='{$target_id}'
                                                     AND email_addr_bean_rel.bean_module='{$target_type}'");
                            $GLOBALS['log']->info("Notice: {$target_type} :: {$target_id} email was marked as invalid");
                        };
                    // #ADDED# 
  

this variant log the datetime when email was marked as invalid

[martyp]

Hey M

I added the code suggested and have just started an email campaign. At the moment I have some 120 odd emails returned for a number of reasons (auto reply, out of office etc) One of the emails had the following (edited out email addresses)

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

blahblah@blah.com
SMTP error from remote mail server after RCPT TO:<blahblah@blah.com>:
host act-mx.blahblah.com [111.111.111.111]: 550 #5.1.0 Address rejected.

------ This is a copy of the message, including all the headers. ------

Return-path: <info@myblah.com>
Received: from localhost ([127.0.0.1])
by cp14.blahblah.com.au with esmtp (Exim 4.69)
(envelope-from <info@myblah.com>)
id 1SC0ES-003KNB-F8
for blahblah@blah.com; Mon, 26 Mar 2012 14:04:32 +1100
Date: Mon, 26 Mar 2012 14:04:32 +1100
To: "Dr. Blah Blah" <blahblah@blah.com>
From: Blah <info@blah.com>
Reply-to: My Blah <corped@otherblah>
Subject: Blah Blah
Message-ID: <fc19065c833a8e90c6c156f8740129a1@localhost>
X-Priority: 3
X-Mailer: PHPMailer (phpmailer.codeworxtech.com) [version 2.3]
X-CampTrackID: 6a5d07ec-ff22-282f-85e8-4f6fdc773ef4
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="b1_fc19065c833a8e90c6c156f8740129a1"
X-DigitalPacific-MailScanner-Information: Please contact the ISP for more information
X-DigitalPacific-MailScanner-ID: 1SC0ES-003KNB-F8
X-DigitalPacific-MailScanner: Not scanned: please contact your Internet E-Mail
Service Provider for details
X-DigitalPacific-MailScanner-SpamCheck:
X-DigitalPacific-MailScanner-From: blah@blah.com
X-Spam-Status: No


--b1_fc19065c833a8e90c6c156f8740129a1
Content-Type: text/plain; charset = "UTF-8"
Content-Transfer-Encoding: quoted-printable

It hasn't been processed as a bounced email, (had it's email address set to invalid) Could you tell me if you see anything that sticks out as to why this hasn't been processed?

Thanks for your time, help and patience