Strange Problems with Zimbra Web Interface

[CrypTom]

Hi all

We've been using Zimbra for years and it always worked reliably until I upgraded from 6.0.7 to 6.0.10 three months ago.

Users started reporting problems with the Zimbra web interface approximately once a week. The effects were strange as not all users were affected.

Problems reported from users and seen with my own eyes were (it is always just the web ui that is affected!):
- not possible to login (while IMAP, etc. works) or
- not possible to create a calendar entry or
- not possible to create a new email message or
- admin web ui does not load (not even the login screen)

Restarting Zimbra seems to fix the problems.

We hoped for a fix when we upgraded to 6.0.12, and again when we applied the 6.0.12 patch. But instead of an improvement we now have to restart Zimbra at least once a day as the problems keep reappearing more often.

I searched the logs and the only thing I found were errors in /opt/zimbra/log/mailbox.log. For example, this morning I was not able to login to the admin web interface, while logging into the normal web interface worked fine. I got the following error every time I tried to load the admin ui:

2011-06-01 07:19:03,261 INFO  [btpool0-699://localhost:7071/service/admin/soap/GetDomainInfoRequest] [ip=127.0.0.1;] soap - GetDomainInfoRequest
2011-06-01 07:19:03,267 WARN  [btpool0-701://zimbra.mydomain.com:7071/zimbraAdmin/] [] log - /zimbraAdmin/: java.io.IOException: Permission denied
2011-06-01 07:19:03,268 WARN  [btpool0-701] [] log - /zimbraAdmin/
java.io.IOException: Permission denied
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:734)
        at org.mortbay.io.nio.DirectNIOBuffer.<init>(DirectNIOBuffer.java:70)
        at org.mortbay.jetty.servlet.DefaultServlet$NIOResourceCache.fill(DefaultServlet.java:1000)
        at org.mortbay.jetty.ResourceCache.load(ResourceCache.java:198)
        at org.mortbay.jetty.ResourceCache.lookup(ResourceCache.java:166)
        at org.mortbay.jetty.servlet.DefaultServlet.doGet(DefaultServlet.java:411)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:705)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:814)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:218)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:422)
        at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:330)
        at org.mortbay.jetty.servlet.Dispatcher.error(Dispatcher.java:135)
        at org.mortbay.jetty.servlet.ErrorPageErrorHandler.handle(ErrorPageErrorHandler.java:129)
        at org.mortbay.jetty.Response.sendError(Response.java:274)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:475)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:218)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:422)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.handler.rewrite.RewriteHandler.handle(RewriteHandler.java:230)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.handler.DebugHandler.handle(DebugHandler.java:77)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:543)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:929)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:405)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
        at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)

I restarted Zimbra and the admin interface worked again. Two hours later I got a phone call reporting that no calendar events could be created. I checked the same log:

2011-06-01 09:20:53,041 WARN  [btpool0-84://zimbra.mydomain.com/zimbra/js/CalendarAppt_all.js.zgz?v=110306011814] [] log - /zimbra/js/CalendarAppt_all.js.zgz: java.io.IOException: Permission denied

Permission denied again.

I stopped Zimbra, executed zmfixperms as root and started Zimbra again. The next phone call came not even an hour later: "I cannot create a new mail". The log shows:

2011-06-01 10:01:17,029 WARN  [btpool0-43://zimbra.mydomain.com/zimbra/js/Mail_all.js.zgz?v=110306011814] [] log - /zimbra/js/Mail_all.js.zgz: java.io.IOException: Permission denied

Again, restarting Zimbra fixed the problem.

No one has reported errors with IOException and Permission denied in this forum during the last couple of years.

Any ideas what could be responsible for this really annoying behavior?

Thanks in advance
CrypTom

[emiro]

Well, I'm stumped. (Thought I'd give you a new perspective... but I haven't come up with much.) >< Best thing I've got for you is that we've been having a *lot* of java-related issues over the past month. Particularly in firefox. Corrupted java on your clients might be to blame (would explain why server upgrades and permissions resets haven't worked). Have you tried multiple clients? (IE, chrome, firefox, safari).

[CrypTom]

Thanks, emiro, for your advice.

It has been a week without the annoying "Permission denied" Exception. But today it came back again.

The problem does not seem to be appearing only on specific browsers or operating systems. I've tested Firefox on Linux and Internet Explorer on Windows, while users reported the problem while using Chrome and Firefox under Windows.

Any ideas?

Thanks
CrypTom

[emiro]

Nope, and we're still trying to figure out our Zimbra/firefox issues. Thought it was the presence of "Java Console" that was causing it, but the trouble remains. If we resolve ours I'll report back what we found, in case it might help you.

[CrypTom]

It happened again:

2011-06-14 10:44:17,152 WARN  [btpool0-177://zimbra.mydomain.com/zimbra/js/Tasks_all.js.zgz?v=110306011814] [] log - /zimbra/js/Tasks_all.js.zgz: java.io.IOException: Permission denied
2011-06-14 10:44:37,266 WARN  [btpool0-177://zimbra.mydomain.com/zimbra/] [] log - /zimbra/: java.io.IOException: Permission denied
2011-06-14 10:44:37,347 WARN  [btpool0-169://zimbra.mydomain.com/zimbra/img/dwt/ImgWarning_32.gif] [] log - /zimbra/img/dwt/ImgWarning_32.gif: java.io.IOException: Permission denied
2011-06-14 10:44:58,496 WARN  [btpool0-177://zimbra.mydomain.com/zimbra/] [] log - /zimbra/: java.io.IOException: Permission denied
2011-06-14 10:44:58,531 WARN  [btpool0-177://zimbra.mydomain.com/zimbra/img/dwt/ImgWarning_32.gif] [] log - /zimbra/img/dwt/ImgWarning_32.gif: java.io.IOException: Permission denied
2011-06-14 10:45:38,758 WARN  [btpool0-169://zimbra.mydomain.com/zimbra/yui/2.7.0/animation/animation-debug.js] [] log - /zimbra/yui/2.7.0/animation/animation-debug.js: java.io.IOException: Permission denied
...
lots of images with the same error

This time, I waited with restarting zimbra in order to test a few browsers and operating systems. I can log into zimbra web client under linux with firefox, but not access tasks. On windows I wasn't even able to login from safari, opera and chrome.

But after a few minutes, everything worked as usual (login and tasks are working again) even without restarting zimbra! No more errors in the logs.

Strange...

[CrypTom]

Well, not all error are gone...

If I login to the standard (HTML) version, I get no images and tons of Permission denied errors in the log (as the above).

For example (just one line):

2011-06-14 11:11:20,247 WARN  [btpool0-32://zimbra.mydomain.com/zimbra/img/calendar/ImgCalendarFolder.gif] [] log - /zimbra/img/calendar/ImgCalendarFolder.gif: java.io.IOException: Permission denied

I've checked this one file mentioned above:

user@zimbra:~$ ls -l /opt/zimbra/jetty/webapps/zimbra/img/calendar/ImgCalendarFolder.gif
-rw-r--r-- 1 zimbra zimbra 906 2011-03-06 10:07 /opt/zimbra/jetty/webapps/zimbra/img/calendar/ImgCalendarFolder.gif

Permissions seem ok...

[denson]

I have the same problem, have to restart the Zimbra twice a day, what to do? help !!!

[zadok]

I have the same problem.
Since i migrated to Zimbra Release 7.1.0_GA_3140.RHEL5_64_20110329150833
I have to reboot the service every day.

Every now and then the web interface freeze and the only solution is "zmcontrol restart".

I have to arrange a simple script in crontab to check the login page and restart like this:

#!/bin/sh
cd /tmp
wget Zimbra offers Open Source email server software and shared calendar for Linux and the Mac.
if [ $? -gt 0 ]; then
/opt/zimbra/bin/zmcontrol restart && echo "script zimbra restart" | mail -s "zimbra restart" xxx.xxx@xxx.com
fi
/bin/rm -rf /tmp/index*

any help is welcomed

[phoenix]

Install the latest patch from the ZCS download page.

[denson]

I installed the latest patch, but the situation has not changed, still the same problem.

Release 7.1.1_GA_3196.UBUNTU8 UBUNTU8 FOSS edition.

help somebody, tried several times to /opt/zimbra/libexec/zmfixperms --verbose --extended, still the same problem.