Zimbra 4.5.7+High CPU usage with Thunderbird+Lightning.

[teet]

Hello

Many users(including myself) are having problems with Zimbra 4.5.7(Network Edition). Problem occurs when starting Thunderbird(depending on machine) 1-4 minutes thunderbird is slow and unusable. Same problem occurs when reloading remote calendar.
Same case with Sunbird(High CPU usage, Memory usage about 300-400MB)
Tried 4.5.6 everything works fine(Free edition).
Both systems run on Centos5.
When this slowness occurs server load is about 1-4%.
Outlook connectors (on 4.5.7 NE) are working fine.
Any help would be appreciated.

[dkarp]

Can you turn on IMAP protocol logging and then see what happens when a user logs in via Thunderbird?

[teet]

Thanks for replay.
I was unable to find where i can turn verbose for imap.
I found that lightning bug was fixed in latest stable release(4.5.9)
We will upgrade server to latest stable version.
If any changes occur i'll let know(good or bad)
Ticket from pm.zimbra.com : 19417 normal
Error when publishing shared calendar with Lightning
4.5.9 Calendar - Server VERIFIED 4 jhahm
UPDATE
Edit: Nothing different with 4.5.9 CPU and MEM usage still high.
Tried latest lightning (0.7 official release) everything same.

[f10ga11ma]

I am having the same problem and so are several of my users. Currently we are using 4.5.4 and ever since Lightning 0.5 arrived I personally started experiencing the problem (near 100% CPU and huge memory usage up to 1.3GB) when I first switched to Lightning 0.5. This problem appears to be specific to the user's calendar though as I am able to load another calendar for a shared role account that I use and it has no problems. Lightning 0.7 has the same problem and so does SunBird 0.7. This occurs on Linux, Mac and Windows XP.
Other users in the office have started to report the problem as well.
I am willing to help out with any testing that might help -- but I don't think it is worth exploring verbose modes for IMAP as this problem is independent of Thunderbird and is related directly to Zimbra and Sunbird.
In my case at least the problem appears to be that I have a very strange calendar entry which has around 773,000 lines which look like this: " \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n\n\n\n\n\n\n\n\n\n"

This is the very first calendar entry I have and if I try to view it (or delete it) via the web ui, I get the standard firefox browser popup error about a runaway script.

It seems to me that the calendar entry has been corrupted by something. How can I fix this?

[teet]

Today i tried lightning 0.8pre. It worked just fine.
Only different is, i configured lightning with totally new zimbra accounts.
Later i tried my own account(few months of calendar) same problems!
I had same problem with mac os+zimbra connector ical(system mem and cpu usage very high).
Many our clients are using thunderbird+Lightning and mac iCal. For them there is no point for paying NE Zimbra(Because it's not working, worse it makes computer unusable!)

[f10ga11ma]

I spoke to Zimbra support. There is a workaround to this. I am surprised that they didn't post to this thread as I referenced it in my email. Anyway, the trick is to delete the offending calendar entry. In my case I found it to be the very first calendar entry. I looked at the full ical dump and found that the very first entry had all those "\n" characters in it. Zimbra gave me instructions on deleting that entry. They are paraphrased/corrected below...

You need to get the ID of the calendar appointment. You can do this with Firebug by looking at the post request sent to the server from the web UI (actually in my case, I had to use a packet sniffer and look at the request headers. Maybe I just don't know how to use firebug). It also might show up in the error you are getting as well. The request will look something like this:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
<soap:Header><context xmlns="urn:zimbra">
<userAgent name="ZimbraWebClient - FF2.0 (Mac)" version="4.5.9_GA_1454.MACOSXx86"/>
<sessionId id="57"/><change token="40233" type="new"/><authToken>...</authToken>
<format type="js"/></context></soap:Header><soap:Body>
<GetMsgRequest xmlns="urn:zimbraMail">
<m id="12902-15155" read="1" html="1"/>
</GetMsgRequest></soap:Body></soap:Envelope>

The item id for the calendar item is the number before the "-" in the id property:

<m id="12902-15155" read="1" html="1"/>

So, 12902.

Next, you can use zmmailbox to delete the item:

$ zmmailbox -z -m <mailbox>
mbox> gru -o /tmp/cal /Calendar
mbox> di 12902

After I did this, everything worked perfectly. It is also possible that 4.5.10 addresses this issue, but I don't know as I haven't installed it.

[teet]

Thank you!
This maybe fine for me or some other administrator calendar.
There is no way i could fix this way 100+ calendars.
There must be some tool or script to make fixing that bug more easily!
Anyway give any information about my own calendar.

[glitch23]

Unfortunately there isn't. I had this problem back in June and it can occur in a number of ways but the most common ones are 1) Corrupted Calendar entry (usually froma migration from Exchange) and 2) A very large Calendar.

The second instance has been fixed in the latest lightning Nightly build, Lightning was not releasing the old calendar info from memory when it refreshed the calendar therefore when it refreshed every 60 minutes (default) it would eventually eat up all the ram and CPU your computer had.

The first seems to stem from how a calendar entry is formatted by Outlook and wreaks havoc when trying to retrieve the ical version that was migrated to Zimbra.

[mindunit]

Just wanted to say many thanks for this solution; I found it wasn't really necessary to use Firebug or zmmailbox however.

I did a dump of the ICS and also found around 9 million "\n"s in a particular entry. I made a note of the DTSTAMP: number, found the item in the calendar owner's web interface by date, (in my case DTSTAMP:20080423T085906Z = 23rd April 08) and simply deleted it from there. Everything's been fine in Lightning since.

Maybe I'm missing something important but it seemed easier this way. Obviously no easier to do 100 calendars tho...