[SOLVED] Outlook 2007 Initial Sync Results in Massive zdb

[BenG]

G'day,
We have been struggling with a customer site recently.
They have been working fine for a number of months, but in the past couple of weeks we have been getting Sync issues.
This is becoming critical because it is blowing their 100GB data allocation, and data is expensive here in Oz.

The symptom we get is similar to above, but it is on 5.0.16. (Yes I know we should upgrade, but it's hard to coordinate multiple clients to update their ZCO together.)
We basically have one client, in one office, that keeps getting emails "stuck" in the sync process.
They have the 5.0.16 ZCO on Outlook 2007, and their setup is the same as other staff.
The only difference is they have a big MBox, around 3GB.

Initially their problem started when they were reorganising their mailbox, and shifting a lot of large mails (5-20mb), so I thought that was the problem.
But the sync error still seems to happen with the last one being a 5mb file in their draft. As soon as you delete the offending file, Sync works again.
Receiving emails seems to always work (regardless of the sync issue).
Below is the snip of the log between the base of a message, and the beginning of the error.

I know that the obvious troubleshooting would be to upgrade first and then see if it fixes the issue.
However as this is a shared server with Multiple domains/clients, it's not that simple.

So, is there a way to easily tell which email is getting stuck?
When I review the ZCO logs, they are huge, and it's not always obvious where the problem is. Eg the first errors we had "Upload Failed", while the current one is below. In fact the client themselves can't tell until they check their webmail.

Secondly, is there any impact on having a mismatch between ZCO and Server versions? Say a server at 5.0.19 and a mix of ZCO at 5.0.16?
(I will crosspost the last question and update this thread if someone is able to answer it.)

both/either the Sender and/or Addressee. If you are not the Addressee of =
this email/files please immediately contact the Sender and delete this =
email/files.
    </p>
    <p>
    </p>
  </td></tr></table></body>
</html>

------=_NextPart_000_1F39_01CA5417.C76FE2E0--

12-11-2009 16:37:01.942 [2992]: *2689*
12-11-2009 16:37:01.943 [2992]: *2689* DEBUG: in Zimbra::Rpc::Connection::ReadResponseFully - Total bytes read from wire: 9998
12-11-2009 16:37:01.943 [2992]: *2689* <<<<-------- HTTP stream End (Response) ----------------------------------------------->>>>
12-11-2009 16:37:01.944 [2992]: *2689* Exit Zimbra::Rpc::Connection::DoGet ...
12-11-2009 16:37:01.946 [2992]: Zimbra::Sync::SaxHandlers::InitialSyncSAXHandler::StartMessage. caught unexpected exception:
12-11-2009 16:37:01.946 [2992]: Zimbra::Sync::SaxHandlers::InitialSyncSAXHandler::StartMessage. EXCEPTION INFO: processing item id: 1462
12-11-2009 16:37:01.947 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. No Req RPC -- local failure tempfile not written
12-11-2009 16:37:01.947 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. No Rsp RPC -- local failure tempfile not written
12-11-2009 16:37:01.949 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. trying write 9998 bytes into C:\Users\Dirk\AppData\Local\Temp\Mim-20091112T163701.948-1462.txt
12-11-2009 16:37:01.952 [2992]: Zimbra::Sync::SaxHandlers::InitialSyncSAXHandler::DownloadCreateMessage. the base function failed for mids
12-11-2009 16:37:01.952 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. No Req RPC -- local failure tempfile not written
12-11-2009 16:37:01.953 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. No Rsp RPC -- local failure tempfile not written
12-11-2009 16:37:01.954 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. trying write 9998 bytes into C:\Users\Dirk\AppData\Local\Temp\Mim-20091112T163701.953-1462.txt
12-11-2009 16:37:01.956 [2992]: ERROR: Zimbra::Sync::SaxHandlers::InitialSyncSAXHandler::StartMessage - DownloadCreateMessage failed to init sync message id(1462)
12-11-2009 16:37:01.957 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. No Req RPC -- local failure tempfile not written
12-11-2009 16:37:01.958 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. No Rsp RPC -- local failure tempfile not written
12-11-2009 16:37:01.959 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. trying write 9998 bytes into C:\Users\Dirk\AppData\Local\Temp\Mim-20091112T163701.958-1462.txt
12-11-2009 16:37:01.961 [2992]: ERROR: Zimbra::Sync::SaxHandlers::InitialSyncSAXHandler::StartMessage - DownloadCreateMessage failed to init sync message id(1462)
12-11-2009 16:37:01.962 [2992]: Zimbra::Mapi::ZimbraMailItem::ZimbraMailItemType. Failed to retreive Object type and message class. Error 0x80040119
12-11-2009 16:37:01.963 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::DownloadCreateMessage. parent folder for message id: 454 is not really a folder, abort! probably a reserved folder
12-11-2009 16:37:01.963 [2992]: Zimbra::Sync::SaxHandlers::InitialSyncSAXHandler::DownloadCreateMessage. the base function failed for mids
12-11-2009 16:37:01.964 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. No Req RPC -- local failure tempfile not written
12-11-2009 16:37:01.964 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. No Rsp RPC -- local failure tempfile not written
12-11-2009 16:37:01.966 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. trying write 9998 bytes into C:\Users\Dirk\AppData\Local\Temp\Mim-20091112T163701.965-454.txt
12-11-2009 16:37:01.968 [2992]: ERROR: Zimbra::Sync::SaxHandlers::InitialSyncSAXHandler::StartMessage - DownloadCreateMessage failed to init sync message id(454)
12-11-2009 16:37:01.969 [2992]: Zimbra::Mapi::ZimbraMailItem::ZimbraMailItemType. Failed to retreive Object type and message class. Error 0x80040119
12-11-2009 16:37:01.970 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::DownloadCreateMessage. parent folder for message id: 455 is not really a folder, abort! probably a reserved folder
12-11-2009 16:37:01.970 [2992]: Zimbra::Sync::SaxHandlers::InitialSyncSAXHandler::DownloadCreateMessage. the base function failed for mids
12-11-2009 16:37:01.971 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. No Req RPC -- local failure tempfile not written
12-11-2009 16:37:01.971 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. No Rsp RPC -- local failure tempfile not written
12-11-2009 16:37:01.973 [2992]: Zimbra::Sync::SaxHandlers::SyncSAXHandler::WriteSoapToTempFile. trying write 9998 bytes into C:\Users\Dirk\AppData\Local\Temp\Mim-20091112T163701.972-455.txt
12-11-2009 16:37:01.975 [2992]: ERROR: Zimbra::Sync::SaxHandlers::InitialSyncSAXHandler::StartMessage - DownloadCreateMessage failed to init sync message id(455)
12-11-2009 16:37:01.976 [2992]: Zimbra::Mapi::ZimbraMailItem::ZimbraMailItemType. Failed to retreive Object type and message class. Error 0x80040119

[fsiegel]

Here are a few answers:

1) It is very possible that upgrading would solve your problem. Check out bug 40396. This was fixed in 5.0.20. ZCO continually tried to upload emails with large attachments because it wanted to put them in the Sent folder. You should consider upgrading.

2) As far as mismatched ZCO clients and ZCS servers: you are generally ok if ZCO is older than the server. You should always upgrade your servers first. We don't support newer ZCO's with older servers. Of course it's best to be matched up, and back around 5.0.7, there were some enhancements that happened on the server side that required a corresponding ZCO. But I think from 5.0.16 to 5.0.19 there weren't any of those issues. Having a 5.0.16 ZCO against a 5.0.19 server should be OK. But 5.0.16 is 4 versions old -- that's getting a bit long in the tooth.

3) To find offending mail messages: well it looked like the log said that the one with Zimbra Id 1462 was a problem (there were also some others [454 and 455]. These errors are coming from the SaxHandler, though, which means that they are being synced down from the server. To see which message 1462 is, for example, you can hit the content servlet:

http[s]://<yourserver>:<yourport>/service/content/get?id=1462&fmt=sync

If the message is already in Outlook, you can also enable the Support bar and do Open By Zimbra Id.

To find offending messages going from Outlook to Zimbra, you'd have to look in the logs and see where the processing of change records gets hung up. You can look for "processing change record", etc. But this is a little tougher.
If your editing an existing message in Outlook, then it would have a Zimbra Id, and you could use the techniques above. If it's new, then it won't, and you'll have to wade through the log file.

[BenG]

G'day,
Thanks for such a great response.
I'll look at all of the items covered and report back anything I find.
Much appreciated.

Ben.

[BenG]

Ok, just to save anyone else the feeling of dread if they follow:

http[s]://<yourserver>:<yourport>/service/content/get?id=1462&fmt=sync

Just remember that FF is a single process app;
ie if you opened a new FF and just happened to have a Zimbra Webmail client open and logged in on another, it WILL just open the email. No login prompt etc, and obviously it will be one of the logged in user.
I chose a random# and just happened to open a very sensitive email
Then spent 30 seconds panicking that just ANY user could do this.... until I realised my mistake.....

Ben

[jurness]

Originally Posted by fsiegel

This seems like a server error to me. Undoubtedly a hassle to go back to a 6.0.1 server unless you've got an installation around somewhere, but the fact that it worked with 6.0.1, and then not with 6.0.2, and then when you went back with just 6.0.1 ZCO, it failed, sounds like a server issue.

It also sounds like ZCO needs to handle whatever this error is in a better way.

I need to come up with a way to repro this in-house.

Are you saying that if I run the 6.0.1 installer, it will seamlessly downgrade to that version? I'd rather avoid that but can do it if necessary.

Is there a way to start with a clean slate with Outlook/ZCO. I mean if 6.0.1 worked at one point it seems like it ought to work again. Like I said, I do have a user on Outlook 2007/ZCO 6.0.1/Windows7 and it is working flawlessly.

How can I start with a clean slate? Uninstallation is obviously not totally clean.

I just started deploying this a few weeks ago and this is putting a serious damper on migrating my users.

[fsiegel]

Not sure about the server installer, but I wouldn't mess with that -- probably best to keep your 6.0.2 server.

As far as a clean slate with ZCO, you should uninstall ZCO 6.0.2. To be safe, you can go into regedit and delete HKcu\Software\Zimbra. Then you should delete your existing Zimbra profiles. Then install ZCO 6.0.1 again. That might work.

I hope to have more info about the server error today.

[fsiegel]

Note from a server guy on the HTTP 500 error:

"Looks like a permission issue on the mailbox and likely not an issue with the formatter itself. Can you access the REST URL with zmmailbox or curl?"

[jurness]

I'll need that question in layman's terms...

[fsiegel]

zmmailbox is a Zimbra admin command for mailboxes

Zmmailbox - Zimbra :: Wiki

curl is a command line tool for transferring files with URL syntax

We just want to know if there is some permission problem.

As far as why the zdb grows, it could be all the local failures we're putting out. We fixed a bug (39687) that detects the over quota condition and prevents repeated failures. We undoubtedly should do the same for this.

If I could find the exact repro steps for your problem, we could fix this.

[jurness]

Bug 42393 &ndash; Outlook Doesn't Sync All Mail -- creates massive zdb