"Trashed" messages are included in unread/total message count for other folders

[Morac]

I installed Zimbra beta 3 which is the first Zimbra I've ever used and it appeared to be syncing fine, but it constantly told me I had new messages in my Inbox when there weren't any new messages there.

I also noticed that the message count for the Inbox folder was way off. If I hovered over the Inbox folder text in Yahoo Mail it said I had 63 messages, which was correct. If I did the same in Zimbra it said I had 149 messages which was wrong. The right pane only displayed the 63 messages which were in my Inbox. It showed message 1-63 at the top right of the messages list in message view and 1-62 in conversation view. The last is another bug since it should have said either 1-63 or if it's only counting conversations, something like 1-50.

Anyway I reset the account daya and it started to sync the Inbox again and I noticed that a number of messages that showed up in the Inbox were messages that I had already deleted earlier today via the Yahoo webmail interface. These messages are in the "Trash" folder. These messages remained in Zimbra's Inbox message list until I switched to a different folder and back and then they disappeared.

I found the same problem occurs with other folders from which I deleted messages that are still in the "Trash" folder. Since I filter my messages into different folders and subsequently delete most of them when I'm done, the total message count is off on many of my folders.

I tried syncing the "Trash" folder which lists the correct number of messages, but even after that the mail count on the other folders were still wrong. In fact the mail count on the Inbox went up because I had just deleted some more messages that came into my Inbox and after syncing zimbra includes these new "Trash" messages in the Inbox message count.

What's worse is that it's apparently counting one of the "Trash" messages as "unread" even though none of them are since the unread count on my Inbox is off by one.

I'm assuming this is not normal. Is there a way to fix this?

[Morac]

I think I'm understanding what's happening here from looking at the log files. This is the log entry for selecting the INBOX (UID modified):

C: C06 SELECT Inbox
S: * 149 EXISTS
S: * 0 RECENT
S: * OK [UNSEEN 81] Message 81 is first unseen
S: * OK [UIDVALIDITY 1] UIDs valid
S: * OK [UIDNEXT XXXX33] Predicted next UID
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft)] Permanent flags
S: C06 OK [READ-WRITE] SELECT completed; now in selected state

Apparently, when polling Yahoo Mail via IMAP4 it will include deleted messages in the response. Yahoo Mail returns a Inbox message count of 149. So Yahoo stores delete messages in both the Trash folder and the original folder.

Zimbra is still bugged though because when it requests a fetch on all 149 messages, the ones that have been deleted have the "\Deleted" flag set and therefore should not be counted.

Here's 5 fetches that occurred on my Inbox (UIDs modified):

C: C73 UID FETCH XXXX01,XXXX50,XXXX33,XXXX77,XXXX02 (FLAGS INTERNALDATE)
S: * 55 FETCH (FLAGS (\Seen) INTERNALDATE "16-Jul-2008 23:57:16 -0700" UID XXXX02)
S: * 56 FETCH (FLAGS (\Seen) INTERNALDATE "17-Jul-2008 15:26:04 -0700" UID XXXX77)
S: * 57 FETCH (FLAGS (\Seen) INTERNALDATE "21-Jul-2008 17:28:18 -0700" UID XXXX33)
S: * 58 FETCH (FLAGS (\Deleted \Seen) INTERNALDATE "22-Jul-2008 01:21:29 -0700" UID XXXX50)
S: * 59 FETCH (FLAGS (\Deleted \Seen) INTERNALDATE "22-Jul-2008 14:26:55 -0700" UID XXXX01)

These are 5 messages "in" the Inbox, but 2 of them have been deleted so are actually in the "Trash" folder and should be ignored.

So the Zimbra client can't just display what was received in the SELECT command. It needs to keep a tally on the responses to the FETCH's and not count any that have been "\Deleted".

I'm using the new Yahoo Mail, not the classic version if that makes a difference.

edit:

Okay I found a "fix" for this. Yahoo's servers will include deleted items in a folder via the IMAP4 SELECT command until a message is deleted from that folder using an IMAP4 client (I'm not sure what command is used, but it looks like it APPENDS to the trash). At this point the folder count will no longer include deleted items.

C: C19 SELECT Inbox
S: * 63 EXISTS
S: * 0 RECENT
S: * OK [UNSEEN 0] Message 0 is first unseen
S: * OK [UIDVALIDITY 1] UIDs valid
S: * OK [UIDNEXT XXXX45] Predicted next UID
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft)] Permanent flags
S: C19 OK [READ-WRITE] SELECT completed; now in selected state

This seems to point towards some kind of initialization problem with Yahoo's IMAP server with respect to folders.

So the work around if you have this problem is to delete a message from each folder where the count is wrong and then sync. Wait about 30 seconds and sync again and the message counts will be correct.

Note you can move messages from the trash into the problem folders, sync and then delete those messages. That way you don't have to delete any messages you want to keep.

[Morac]

One last update. Apparently the work around only works if you stop using the Yahoo Mail web page. If you delete any messages using the web page the count will get out of whack again and you'll need to repeat the work around above.

It seems there is some kind of incompatibility between how IMAP4 treats a delete and how Yahoo web mail interface does a delete. The web interface is okay with IMAP4 deletes, but not the other way around.

[jjzhuang]

you are exactly right. there's already a bug open for this:

Bug 30267 – "Trashed" messages are included in unread/total message count for other folders