Accented characters shown as question marks

[fcolpron]

We currently are planning a migration from Netmail to Zimbra and came across a small issue: lots of users read their emails with Thunderbird through an IMAP connection to Netmail, and we plan to activate the IMAP feature on Zimbra so these users can keep their mail environment.

However, some accented characters often used in French show up as question marks in both the Subject and From fields, and less often, in the body of the message itself, when read from Thunderbird.

When using Thunderbird to access the messages on Netmail, the messages are not affected.

Everything looks fine on Zimbra WebUI.

If we move a message from Netmail (that is correctly displayed) to Zimbra, accents become ?. If we move these messages back to Netmail, they show up normally. We merely drag and drop the messages from an account to another in Thunderbird. So messages are unaffected.

Also, accented characters show as question marks in the message listing view in Thunderbird, but are correctly shown in Thunderbird preview pane.

We found out that messages that have been sent with encoding data in the Subject and From fields don't have this problem. Encoding information is shown this way (quoted from memory): ?=ISO-8859-1?Here's the subject.

I could reproduce this behavior with Opera and Pegasus mail clients, so I confirm this is not client related.

Examples of question marked characters: é è à ç

As we just convinced a major customer of ours to switch to Zimbra, we don't want them to change their mind because of this.

I can provide screenshots and examples if required.

Moderators: I wasn't sure if I should post this in Migration, Administration of Translation. Feel free to move it as you see fit.

[dkarp]

Let's see a sample "Show Original" of a message exhibiting this problem.

[fcolpron]

Here's the header taken from Thunderbird when connecting to Zimbra using Imap:

Subject:
à à à à à ö ö ô ô ì é
From:
Fcolpron <fcolpron@domain.com>
Date:
Thu, 24 Jul 2008 08:14:41 -0400 (EDT)
To:
fcolpron <fcolpron@domain.com>
X-Auth-OK:
fcolpron@domain.com
X-NIMS-Flags:
1 1216901681
Return-Path:
fcolpron@domain.com
Received:
from zimbra.domain.com (LHLO zimbra.domain.com) (192.168.254.242) by zimbra.domain.com with LMTP; Thu, 24 Jul 2008 08:16:08 -0400 (EDT)
Received:
from localhost (localhost [127.0.0.1]) by zimbra.domain.com (Postfix) with ESMTP id 0F5E2204957B for <fcolpron@domain.com>; Thu, 24 Jul 2008 08:16:08 -0400 (EDT)
X-Quarantine-ID:
<ABI2AFNntV41>
X-Virus-Scanned:
amavisd-new at
X-Amavis-Alert:
BAD HEADER, Non-encoded 8-bit data (char C3 hex): Subject: \303\240 \303\240 \303\240 \303\240 \303\240 \303[...]
X-Spam-Flag:
NO
X-Spam-Score:
-3.29
X-Spam-Status:
No, score=-3.29 tagged_above=-10 required=6.6 tests=[ALL_TRUSTED=-1.8, AWL=-0.478, BAYES_00=-2.599, SUBJECT_NEEDS_ENCODING=0.001, SUBJ_ILLEGAL_CHARS=1.586]
Received:
from zimbra.domain.com ([127.0.0.1]) by localhost (zimbra.domain.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ABI2AFNntV41 for <fcolpron@domain.com>; Thu, 24 Jul 2008 08:16:07 -0400 (EDT)
Received:
from smtp.domain.com (smtp.domain.com [192.168.254.208]) by zimbra.domain.com (Postfix) with ESMTP id D16CC2049579 for <fcolpron@domain.com>; Thu, 24 Jul 2008 08:16:07 -0400 (EDT)
Received:
from mail by smtp.domain.com with spam-scanned (Exim 4.41) id 1KLymo-0005Un-EL for fcolpron@domain.com; Thu, 24 Jul 2008 07:15:07 -0400
Received:
from [192.168.254.242] (helo=zimbra.domain.com) by smtp.domain.com with esmtp (Exim 4.41) id 1KLymO-0005US-0X for francois.colpron@domain.com; Thu, 24 Jul 2008 07:14:40 -0400
Message-ID:
<1933084795.19991216901681142.JavaMail.root@zimbra .domain.com.com>
MIME-Version:
1.0
Content-Type:
text/plain; charset=utf-8
Content-Transfer-Encoding:
7bit
X-Originating-IP:
[192.168.254.25]
X-Mailer:
Zimbra 5.0.6_GA_2313.SLES10_64 (ZimbraWebClient - FF3.0 (Win)/5.0.6_GA_2313.SLES10_64)
X-JANUS-SPAMCheck:
by smtp.domain.com on Thu, 24 Jul 2008 07:15:06 -0400
X-DSPAM-Result:
Whitelisted
X-DSPAM-Confidence:
0.9997
X-DSPAM-Probability:
0.0000
X-DSPAM-Signature:
4888643a211248493262811
X-DSPAM-Factors:
27, From*fcolpron, 0.00010, Téléphone+poste, 0.00010, From*Colpron+fcolpron, 0.00010, From*Francois Colpron <fcolpron@domain.com>, 0.00010, X-Mailer*Zimbra+GA, 0.00010, X-Mailer*ZimbraWebClient, 0.00010, From*Francois+Colpron, 0.00010, X-Mailer*SLES10, 0.00010, X-Mailer*SLES10, 0.00010, X-JANUS-SPAMCheck*Thu+Jul, 0.00010, Colpron+Domain, 0.00010, Date*Thu+Jul, 0.00010, X-Mailer*SLES10+ZimbraWebClient, 0.00010, Received*192.168.254.242, 0.00010, inc+Téléphone, 0.00010, X-Mailer*GA, 0.00010, X-Mailer*GA, 0.00010, From*Francois, 0.00010, From*fcolpron+domain, 0.00010, X-Mailer*Zimbra, 0.00010, Téléphone, 0.00010, X-Mailer*GA+SLES10, 0.00010, X-Mailer*GA+SLES10, 0.00010, Domain, 0.00010, Domain+inc, 0.00010, X-Mailer*Win, 0.00020, X-Mailer*Win+GA, 0.00020


Email body

Subject appears normally in Thunderbird's view pane, but not in messages listing.

For security reasons, our domain name has been changed to domain.com. No other info have been touched.

Now, the headers taken from Zimbra WebUI, looks slightly different, Subject shows up good in Zimbra WebUI.

X-Auth-OK: fcolpron@domain.com
X-NIMS-Flags: 1 1216901681
Return-Path: fcolpron@domain.com
Received: from zimbra.domain.com (LHLO zimbra.domain.com)
(192.168.254.242) by zimbra.domain.com with LMTP; Thu, 24 Jul 2008
08:16:08 -0400 (EDT)
Received: from localhost (localhost [127.0.0.1])
by zimbra.domain.com (Postfix) with ESMTP id 0F5E2204957B
for <fcolpron@domain.com>; Thu, 24 Jul 2008 08:16:08 -0400 (EDT)
X-Quarantine-ID: <ABI2AFNntV41>
X-Virus-Scanned: amavisd-new at
X-Amavis-Alert: BAD HEADER, Non-encoded 8-bit data (char C3 hex): Subject:
\303\240 \303\240 \303\240 \303\240 \303\240 \303[...]
X-Spam-Flag: NO
X-Spam-Score: -3.29
X-Spam-Level:
X-Spam-Status: No, score=-3.29 tagged_above=-10 required=6.6
tests=[ALL_TRUSTED=-1.8, AWL=-0.478, BAYES_00=-2.599,
SUBJECT_NEEDS_ENCODING=0.001, SUBJ_ILLEGAL_CHARS=1.586]
Received: from zimbra.domain.com ([127.0.0.1])
by localhost (zimbra.domain.com [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id ABI2AFNntV41 for <fcolpron@domain.com>;
Thu, 24 Jul 2008 08:16:07 -0400 (EDT)
Received: from smtp.domain.com (smtp.domain.com [192.168.254.208])
by zimbra.domain.com (Postfix) with ESMTP id D16CC2049579
for <fcolpron@domain.com>; Thu, 24 Jul 2008 08:16:07 -0400 (EDT)
Received: from mail by smtp.domain.com with spam-scanned (Exim 4.41)
id 1KLymo-0005Un-EL
for fcolpron@domain.com; Thu, 24 Jul 2008 07:15:07 -0400
Received: from [192.168.254.242] (helo=zimbra.domain.com)
by smtp.domain.com with esmtp (Exim 4.41)
id 1KLymO-0005US-0X
for francois.colpron@domain.com; Thu, 24 Jul 2008 07:14:40 -0400
Date: Thu, 24 Jul 2008 08:14:41 -0400 (EDT)
From: Francois Colpron <fcolpron@domain.com>
To: francois <francois.colpron@domain.com>
Message-ID: <1933084795.19991216901681142.JavaMail.root@zimbra .domain.com>
Subject: à à à à à ö ö ô ô ì é
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [192.168.254.25]
X-Mailer: Zimbra 5.0.6_GA_2313.SLES10_64 (ZimbraWebClient - FF3.0 (Win)/5.0.6_GA_2313.SLES10_64)
X-JANUS-SPAMCheck: by smtp.domain.com on Thu, 24 Jul 2008 07:15:06 -0400
X-DSPAM-Result: Whitelisted
X-DSPAM-Confidence: 0.9997
X-DSPAM-Probability: 0.0000
X-DSPAM-Signature: 4888643a211248493262811
X-DSPAM-Factors: 27,
From*fcolpron, 0.00010,
Téléphone+poste, 0.00010,
From*Colpron+fcolpron, 0.00010,
From*Francois Colpron <fcolpron@domain.com>, 0.00010,
X-Mailer*Zimbra+GA, 0.00010,
X-Mailer*ZimbraWebClient, 0.00010,
From*Francois+Colpron, 0.00010,
X-Mailer*SLES10, 0.00010,
X-Mailer*SLES10, 0.00010,
X-JANUS-SPAMCheck*Thu+Jul, 0.00010,
Colpron+Domain, 0.00010,
Date*Thu+Jul, 0.00010,
X-Mailer*SLES10+ZimbraWebClient, 0.00010,
Received*192.168.254.242, 0.00010,
inc+Téléphone, 0.00010,
X-Mailer*GA, 0.00010,
X-Mailer*GA, 0.00010,
From*Francois, 0.00010,
From*fcolpron+domain, 0.00010,
X-Mailer*Zimbra, 0.00010,
Téléphone, 0.00010,
X-Mailer*GA+SLES10, 0.00010,
X-Mailer*GA+SLES10, 0.00010,
Domain, 0.00010,
Domain+inc, 0.00010,
X-Mailer*Win, 0.00020,
X-Mailer*Win+GA, 0.00020



Email body

[fcolpron]

We are running ZCS 5.0.6 on Suse Linux Enteprise Server 10.

[fcolpron]

Dan, I think my issue can be related to bug #24808. Is it right?

Bug 24808 &ndash; Charset problems in IMAP

[dkarp]

Might be. Please attach the "Show Origianl" of a sample message to that bug...