ldap died

[maumar]

Centos 4.6 with zcs NE 5.0.11
suddenly and without any reason today ldap has died:

2008-12-17 18:17:49,044 WARN  [ImapServer-50181] [name=xxxxx@xxx.it;mid=13;ip=192.168.100.148;] imap - error checking account status; dropping connection
com.zimbra.common.service.ServiceException: system failure: ZimbraLdapContext
ExceptionId:ImapServer-50181:1229534269043:e24bf7f153ae7690
Code:service.FAILURE
        at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:253)
        at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init>(ZimbraLdapContext.java:304)
        at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init>(ZimbraLdapContext.java:275)
        at com.zimbra.cs.account.ldap.LdapProvisioning.getAccountByQuery(LdapProvisioning.java:457)
        at com.zimbra.cs.account.ldap.LdapProvisioning.getAccountById(LdapProvisioning.java:489)
        at com.zimbra.cs.account.ldap.LdapProvisioning.get(LdapProvisioning.java:513)
        at com.zimbra.cs.account.ldap.LdapProvisioning.get(LdapProvisioning.java:502)
        at com.zimbra.cs.imap.ImapCredentials.getAccount(ImapCredentials.java:89)
        at com.zimbra.cs.imap.ImapHandler.checkAccountStatus(ImapHandler.java:215)
        at com.zimbra.cs.imap.TcpImapHandler.processCommand(TcpImapHandler.java:118)
        at com.zimbra.cs.tcpserver.ProtocolHandler.processConnection(ProtocolHandler.java:160)
        at com.zimbra.cs.tcpserver.ProtocolHandler.run(ProtocolHandler.java:128)
        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Thread.java:595)
Caused by: javax.naming.CommunicationException: mailz.fbs.it:389 [Root exception is java.net.ConnectException: Connection refused]
        at com.sun.jndi.ldap.Connection.<init>(Connection.java:194)
        at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:118)
        at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1578)
        at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2596)
        at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:283)
        at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:175)
        at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:193)
        at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:136)
        at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:66)
        at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)
        at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:247)
        at javax.naming.InitialContext.init(InitialContext.java:223)
        at javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:134)
        at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init>(ZimbraLdapContext.java:290)

it was enough to zmcontrol stop/start to resuscitate it.

[tonster]

That stacktrace just says that mailboxd was having trouble because ldap was down, which you knew. Check out /var/log/zimbra.log for more info on why ldap really died.

[maumar]

you are right, but sadly i set no ldap verbosity at all, in fact:

Dec 17 18:16:49 mailz postfix/smtpd[9926]: NOQUEUE: reject: RCPT from unknown[190.252.232.44]: 554 5.7.1 Service unavailable; Client host [190.252.232.44] blocked using cbl.abuseat
.org; Blocked - see http://cbl.abuseat.org/lookup.cgi?ip=190.252.232.44; from=<biand@diplomatie.gouv.fr> to=<xxxx@xxx.it> proto=ESMTP helo=<[190.252.232.44]>
Dec 17 18:16:50 mailz postfix/smtpd[9926]: lost connection after DATA from unknown[190.252.232.44]
Dec 17 18:16:50 mailz postfix/smtpd[9926]: disconnect from unknown[190.252.232.44]
Dec 17 18:16:54 mailz sshd(pam_unix)[11751]: session opened for user root by (uid=0)
Dec 17 18:17:02 mailz sshd(pam_unix)[11751]: session closed for user root
Dec 17 18:17:15 mailz postfix/smtpd[9396]: timeout after DATA from cp-out11.libero.it[212.52.84.111]
Dec 17 18:17:15 mailz postfix/smtpd[9396]: disconnect from cp-out11.libero.it[212.52.84.111]
Dec 17 18:17:26 mailz slapd[16754]: connection_read(34): no connection!
Dec 17 18:17:41 mailz zimbramon[23561]: 23561:info: zmmtaconfig: Skipping Global system configuration update.
Dec 17 18:17:41 mailz zimbramon[23561]: 23561:info: zmmtaconfig: gacf ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException mailz.f
bs.it:389)
Dec 17 18:17:45 mailz zimbramon[23561]: 23561:info: zmmtaconfig: Skipping All Reverse Proxy URLs update.
Dec 17 18:17:45 mailz zimbramon[23561]: 23561:info: zmmtaconfig: Skipping getAllReverseProxyURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.Com
municationException mailz.fbs.it:389)
Dec 17 18:17:48 mailz zimbramon[23561]: 23561:info: zmmtaconfig: Skipping All Reverse Proxy Backends update.
Dec 17 18:17:48 mailz zimbramon[23561]: 23561:info: zmmtaconfig: Skipping getAllReverseProxyBackends ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming
.CommunicationException mailz.fbs.it:389)
Dec 17 18:17:51 mailz zimbramon[23561]: 23561:info: zmmtaconfig: Skipping All Memcached Servers update.
Dec 17 18:17:51 mailz zimbramon[23561]: 23561:info: zmmtaconfig: Skipping getAllMemcachedServers ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.Com
municationException mailz.fbs.it:389)
Dec 17 18:17:54 mailz zimbramon[23561]: 23561:info: zmmtaconfig: Skipping All MTA Authentication Target URLs update.
Dec 17 18:17:54 mailz zimbramon[23561]: 23561:info: zmmtaconfig: Skipping getAllMtaAuthURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.Communic
ationException mailz.fbs.it:389)

not more than: Dec 17 18:17:26 mailz slapd[16754]: connection_read(34): no connection!

without raising ldap log verbosity i fear nothing can be said
isn't it?

m