weblogon during imapsync: Code:service.FAILURE

[dgsohard]

Currently I migrate a dovecot IMAP system to zimbra using imapsync.

When I try to log on to the adminstratior web page or to a user account using the webclient, the browser hangs and I find the following error message in the opt/zimbra/log/mailbox.log:

com.zimbra.common.service.ServiceException: system failure: ZimbraLdapContext
ExceptionId:btpool0-8://<servername deleted>/service/admin/soap/BatchRequest:1279969554597:7c664e5cce4adb51
Code:service.FAILURE
at com.zimbra.common.service.ServiceException.FAILURE (ServiceException.java:248)
at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init >(ZimbraLdapContext.java:412)
at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init >(ZimbraLdapContext.java:366)
at com.zimbra.cs.account.ldap.LdapProvisioning.getDat aSourcesByQuery(LdapProvisioning.java:5451)
at com.zimbra.cs.account.ldap.LdapProvisioning.getAll DataSources(LdapProvisioning.java:5707)
at com.zimbra.cs.service.admin.GetDataSources.handle( GetDataSources.java:66)
at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEng ine.java:420)
at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:264)
at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:158)
at com.zimbra.soap.SoapServlet.doWork(SoapServlet.jav a:291)
at com.zimbra.soap.SoapServlet.doPost(SoapServlet.jav a:212)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:727)
at com.zimbra.cs.servlet.ZimbraServlet.service(Zimbra Servlet.java:181)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(Ser vletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedCha in.doFilter(ServletHandler.java:1166)
at com.zimbra.cs.servlet.SetHeaderFilter.doFilter(Set HeaderFilter.java:79)
at org.mortbay.jetty.servlet.ServletHandler$CachedCha in.doFilter(ServletHandler.java:1157)
at org.mortbay.servlet.UserAgentFilter.doFilter(UserA gentFilter.java:81)
at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter .java:132)
at org.mortbay.jetty.servlet.ServletHandler$CachedCha in.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(Se rvletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle( SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(Se ssionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(Co ntextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebA ppContext.java:418)
at org.mortbay.jetty.handler.ContextHandlerCollection .handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle (HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(Ha ndlerWrapper.java:152)
at org.mortbay.jetty.handler.rewrite.RewriteHandler.h andle(RewriteHandler.java:230)
at org.mortbay.jetty.handler.HandlerWrapper.handle(Ha ndlerWrapper.java:152)
at org.mortbay.jetty.handler.DebugHandler.handle(Debu gHandler.java:77)
at org.mortbay.jetty.handler.HandlerWrapper.handle(Ha ndlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(Htt pConnection.java:543)
at org.mortbay.jetty.HttpConnection$RequestHandler.co ntent(HttpConnection.java:939)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser. java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpPa rser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnec tion.java:405)
at org.mortbay.io.nio.SelectChannelEndPoint.run(Selec tChannelEndPoint.java:409)
at org.mortbay.thread.BoundedThreadPool$PoolThread.ru n(BoundedThreadPool.java:451)
Caused by: javax.naming.CommunicationException: Bad file descriptor [Root exception is java.net.SocketException: Bad file descriptor]
at com.sun.jndi.ldap.LdapCtx.extendedOperation(LdapCt x.java:3213)
at javax.naming.ldap.InitialLdapContext.extendedOpera tion(InitialLdapContext.java:164)
at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init >(ZimbraLdapContext.java:402)
... 39 more
Caused by: java.net.SocketException: Bad file descriptor
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutp utStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStre am.java:136)
at java.io.BufferedOutputStream.flushBuffer(BufferedO utputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputS tream.java:123)
at com.sun.jndi.ldap.Connection.writeRequest(Connecti on.java:393)
at com.sun.jndi.ldap.LdapClient.extendedOp(LdapClient .java:1172)
at com.sun.jndi.ldap.LdapCtx.extendedOperation(LdapCt x.java:3160)
... 41 more


Only after a "zmmailboxdctl restart" logon works again.

zmcontrol -v says:
Release 6.0.7_GA_2473.DEBIAN5_64 UBUNTU10_64 FOSS edition.

Is there something I can do on zimbra side to prevent this behavior?

How do I prevent users to crash the webserver with an imapsync command during regular operation?

[phoenix]

Originally Posted by dgsohard

zmcontrol -v says:
Release 6.0.7_GA_2473.DEBIAN5_64 UBUNTU10_64 FOSS edition.

Is there something I can do on zimbra side to prevent this behavior?

How do I prevent users to crash the webserver with an imapsync command during regular operation?

It doesn't do this under normal operation. Is this running in a VM or on real hardware and how much RAM is on the Zimbra server? I'd suggest you shutdown Zimbra then run a file system check and see if there's any errors or you can always check some of these threads: site:zimbra.com +"Bad file descriptor" - Yahoo! Search Results

[dalgwen]

Same here.
No VM, no FS error, 2go ram (with mailboxd_java_heap_memory_percent = 25)
It looks like the bug 42870 could be reopened?

But I'm using an unsupported zimbra version + OS , so i don't know if it's relevant.
(Release 6.0.7_GA_2473.DEBIAN5_64 UBUNTU10_64 FOSS edition)

I tried several workaround mentionned in the bug track (ldap timeout, increasing file descriptor limit), but the problem is still here.

I should probably rollback to ubuntu 8.04 with the right build.

[phoenix]

Originally Posted by dalgwen

Same here.
No VM, no FS error, 2go ram (with mailboxd_java_heap_memory_percent = 25)
It looks like the bug 42870 could be reopened?

Not until you can show the problem occuring on a supported system.

Originally Posted by dalgwen

But I'm using an unsupported zimbra version + OS , so i don't know if it's relevant.
(Release 6.0.7_GA_2473.DEBIAN5_64 UBUNTU10_64 FOSS edition)

Why have you installed a Debian version of Zimbra on an unsupported Ubuntu release?

Originally Posted by dalgwen

I should probably rollback to ubuntu 8.04 with the right build.

Yes, that would be my advice.

[dalgwen]

Why have you installed a Debian version of Zimbra on an unsupported Ubuntu release?

Zimbra version for Ubuntu 10.04 is not ready, as it was scheduled for last week, but now rescheduled for the next version. My new dedicated server was installed on ubuntu 10.04 two months ago (Stupid compulsion "take the newer" ), and I was eagerly waiting for the new zimbra version to finish my migration. But I can't wait anymore so I tried version for other OS. Debian 5 version is the closest, as stated elsewhere on this forum and on Internet.
But I know it's completely my fault, I shouldn't use this configuration

Finaly I won't rollback (too much work ; I can use workaround with "zmmailboxdctl restart" and automatic log monitoring; it occurs max once a day).
I will keep an eye on this subject and several other on internet, if anyone has and idea, and wait for a supported version.

Thanks for all.

[dgsohard]

Originally Posted by phoenix

It doesn't do this under normal operation. Is this running in a VM or on real hardware and how much RAM is on the Zimbra server? I'd suggest you shutdown Zimbra then run a file system check and see if there's any errors or you can always check some of these threads: site:zimbra.com +"Bad file descriptor" - Yahoo! Search Results

The file system seems to be okay.

Actually the workaround described in Bug 42870 &ndash; "Bad file descriptor" SocketException causes system failure every several hours solves the problem, i.e. increasing the filedescriptors for root in /etc/security/limits.conf on our system.

I'd expect that this is solved for the coming, official release, which supports Ubuntu 10.04 LTS?

[dalgwen]

Originally Posted by dgsohard

i.e. increasing the filedescriptors for root in /etc/security/limits.conf on our system.

Could you please share some informations about this ?
I added this to the /etc/security/limits.conf :

root soft nofile 524288
root hard nofile 524288

But I think I had the bug meanwhile (not sure).
Did you do something else?

[dgsohard]

Originally Posted by dalgwen

Could you please share some informations about this ?
I added this to the /etc/security/limits.conf :

root soft nofile 524288
root hard nofile 524288

But I think I had the bug meanwhile (not sure).
Did you do something else?

I did exactly this and increased the limits in the /opt/zimbra/jetty/etc/jetty-setuid.xml to 524288 as well, which should not play a role if I understood the comments in the bug report correctly.

On the other hand I don't have a huge load on the system besides the imapsync runs.

[dalgwen]

Originally Posted by dgsohard

I did exactly this and increased the limits in the /opt/zimbra/jetty/etc/jetty-setuid.xml to 524288 as well, which should not play a role if I understood the comments in the bug report correctly.

Thank you for the information, I hope I will not face this error again (one day and a half since last occurence, I cross my finger)