Results 1 to 4 of 4

Thread: LDAP crashing / repeated failure

  1. #1
    geterdone is offline Starter Member
    Join Date
    Jul 2008
    Posts
    1
    Rep Power
    6

    Default LDAP crashing / repeated failure

    A zimbra installation in production that has been running quite well for five months has suddenly started having repeated LDAP failures.

    Nothing was updated just prior to the first failure, certainly not the ldap.conf.in or slapd.conf.in files. Symptoms of the first failure was complete shutdown of all modules except mailbox, which could not immediately resolved by restarting zimbra. Slapd / LDAP was eventually able to load properly, but zimbra would continue to have authentication related errors (e-mail clients repeatedly asking for password, users unable to access webmail, etc). Zmcontrol will report that ldap is Running, but the following pattern shows up in zimbra.log

    Code:
    zimbramon[14376]: 14376:info: zmmtaconfig: Skipping Global system configuration update.
    zimbramon[14376]: 14376:info: zmmtaconfig: gacf ERROR: service.FAILURE (system failure: getDirectContext) (cause: javax.naming.CommunicationException mail.myserver.com:389)
    zimbramon[14376]: 14376:info: zmmtaconfig: Skipping All Reverse Proxy URLs update.
    zimbramon[14376]: 14376:info: zmmtaconfig: Skipping getAllReverseProxyURLs ERROR: service.FAILURE (system failure: getDirectContext) (cause: javax.naming.CommunicationException mail.neuraliq.com:389)
    zimbramon[14376]: 14376:info: zmmtaconfig: Skipping All Memcached Servers update.
    zimbramon[14376]: 14376:info: zmmtaconfig: Skipping getAllMemcachedServers ERROR: service.FAILURE (system failure: getDirectContext) (cause: javax.naming.CommunicationException mail.myserver.com:389)
    zimbramon[14376]: 14376:info: zmmtaconfig: Skipping All MTA Authentication Target URLs update.
    zimbramon[14376]: 14376:info: zmmtaconfig: Skipping getAllMtaAuthURLs ERROR: service.FAILURE (system failure: getDirectContext) (cause: javax.naming.CommunicationException mail.myserver.com:389)
    zimbramon[14376]: 14376:info: zmmtaconfig: Skipping Configuration for server mail.neuraliq.com update.
    zimbramon[14376]: 14376:info: zmmtaconfig: gs:mail.myserver.com ERROR: service.FAILURE (system failure: getDirectContext) (cause: javax.naming.CommunicationException mail.myserver.com:389)
    zimbramon[14376]: 14376:info: zmmtaconfig: Sleeping...Key lookup failed.
    zimbramon[14376]: 14376:info: zmmtaconfig: Skipping Global system configuration update.
    I increased ldap logging by using:

    Code:
    zmlocalconfig -e ldap_log_level=16640
    Since using the slapd debug switch via:

    Code:
    /opt/zimbra/openldap/libexec/slapd -l -d 1 LOCAL0 -4 -u zimbra -h ldap://mail.myserver.com:389 -f /opt/zimbra/conf/slapd.conf
    did not a noticeable effect on logging (unless it's exporting to another log file?).

    The first command, along with increasing zimbra logging to DEBUG starting producing a copious amount of logs to work with.

    I noticed the following failure occurring when users had trouble, not when the regularly-repeating section of logs above occured:
    Code:
    postfix/smtpd[20409]: warning: problem talking to service rewrite: Success
    postfix/smtpd[20382]: warning: problem talking to service rewrite: Connection reset by peer
    postfix/master[21150]: warning: process /opt/zimbra/postfix-2.4.3.4z/libexec/trivial-rewrite pid 22966 exit status 1
    postfix/master[21150]: warning: /opt/zimbra/postfix-2.4.3.4z/libexec/trivial-rewrite: bad command startup -- throttling
    slapd[19956]: conn=7532 op=172 SRCH base="" scope=2 deref=3 filter="(&(|(zimbraMailDeliveryAddress=randomuser@myserver.com)(zimbraMailAlias=randomuser@myserver.com))(objectClass=zimbraAccount))" 
    slapd[19956]: conn=7532 op=172 SEARCH RESULT tag=101 err=0 nentries=1 text= 
    postfix/smtpd[23061]: connect from unknown[<randomIP address>]
    postfix/trivial-rewrite[22994]: error: dict_ldap_connect: Unable to set STARTTLS: -1: Can't contact LDAP server
    zmmailboxdmgr[23141]: status requested
    zmmailboxdmgr[23141]: status OK
    postfix/trivial-rewrite[22994]: error: dict_ldap_connect: Unable to set STARTTLS: -1: Can't contact LDAP server
    postfix/trivial-rewrite[22994]: fatal: ldap:/opt/zimbra/conf/ldap-vad.cf(0,lock|fold_fix): table lookup problem


    For the most part, slapd search requests appear to be going thru since err is 0:
    Code:
    slapd[19956]: conn=7602 op=5 SRCH attr=zimbraMailCanonicalAddress zimbraMailCatchAllCanonicalAddress 
    slapd[19956]: conn=7602 op=5 SEARCH RESULT tag=101 err=0 nentries=0 text= 
    slapd[19956]: conn=7602 op=6 SRCH base="" scope=2 deref=0 filter="(&(|(zimbraMailDeliveryAddress=@somerandomurl.org)(zimbraMailAlias=@somerandomurl.org)(zimbraMailCatchAllAddress=@somerandomurl.org))(zimbraMailStatus=enabled))" 
    slapd[19956]: conn=7602 op=6 SRCH attr=zimbraMailCanonicalAddress zimbraMailCatchAllCanonicalAddress 
    slapd[19956]: conn=7602 op=6 SEARCH RESULT tag=101 err=0 nentries=0 text=
    A deeper analysis of mailbox.log shows at the time of authentication failure:
    Code:
    262 INFO  [IndexWritersSweeperThread] [] index - open index writers sweep: before=1, closed=0, after=1 (0ms)
    201 ERROR [btpool0-57] [ip=<ipofclientconnecting>;ua=DAVKit/2.0 (10.5.3;; wrbt) iCal 3.0.3;] dav - error getting authenticated user
    com.zimbra.common.service.ServiceException: system failure: getDirectContext
    Code:service.FAILURE
            at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:183)
            at com.zimbra.cs.account.ldap.LdapUtil.getDirContext(LdapUtil.java:219)
            at com.zimbra.cs.account.ldap.LdapUtil.getDirContext(LdapUtil.java:203)
            at com.zimbra.cs.account.ldap.LdapProvisioning.getCosByName(LdapProvisioning.java:1749)
            at com.zimbra.cs.account.ldap.LdapProvisioning.get(LdapProvisioning.java:1719)
            at com.zimbra.cs.account.Provisioning.getCOS(Provisioning.java:1539)
            at com.zimbra.cs.account.ldap.LdapProvisioning.makeAccount(LdapProvisioning.java:3339)
            at com.zimbra.cs.account.ldap.LdapProvisioning.getAccountByQuery(LdapProvisioning.java:457)
            at com.zimbra.cs.account.ldap.LdapProvisioning.getAccountByName(LdapProvisioning.java:586)
            at com.zimbra.cs.account.ldap.LdapProvisioning.get(LdapProvisioning.java:503)
            at com.zimbra.cs.account.ldap.LdapProvisioning.get(LdapProvisioning.java:490)
            at com.zimbra.cs.servlet.ZimbraServlet.basicAuthRequest(ZimbraServlet.java:375)
            at com.zimbra.cs.dav.service.DavServlet.service(DavServlet.java:100)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
            at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
            at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
            at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:81)
            at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:148)
            at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
            at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
            at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
            at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
            at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:716)
            at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:406)
            at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
            at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
            at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
            at org.mortbay.jetty.handler.RewriteHandler.handle(RewriteHandler.java:176)
            at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
            at org.mortbay.jetty.Server.handle(Server.java:313)
            at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
            at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:844)
            at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:644)
            at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:205)
            at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
            at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396)
            at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
    Caused by: javax.naming.CommunicationException: mail.myserver.com:389 [Root exception is java.net.UnknownHostException: mail.myserver.com]
            at com.sun.jndi.ldap.Connection.<init>(Connection.java:197)
            at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:118)
            at com.sun.jndi.ldap.LdapClientFactory.createPooledConnection(LdapClientFactory.java:46)
            at com.sun.jndi.ldap.pool.Connections.getOrCreateConnection(Connections.java:185)
            at com.sun.jndi.ldap.pool.Connections.get(Connections.java:126)
            at com.sun.jndi.ldap.pool.Pool.getPooledConnection(Pool.java:129)
            at com.sun.jndi.ldap.LdapPoolManager.getLdapClient(LdapPoolManager.java:310)
            at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1572)
            at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2616)
            at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:287)
            at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:175)
            at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:193)
            at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:136)
            at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:66)
            at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)
            at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:288)
            at javax.naming.InitialContext.init(InitialContext.java:223)
            at javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:134)
            at com.zimbra.cs.account.ldap.LdapUtil.getDirContext(LdapUtil.java:214)
            ... 35 more
    Caused by: java.net.UnknownHostException: mail.myserver.com
            at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
            at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
            at java.net.Socket.connect(Socket.java:519)
            at sun.reflect.GeneratedMethodAccessor167.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at com.sun.jndi.ldap.Connection.createSocket(Connection.java:336)
            at com.sun.jndi.ldap.Connection.<init>(Connection.java:184)
            ... 53 more
    2008-07-25 00:03:10,388 DEBUG [btpool0-57] [] soap - SOAP response:
    <soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
      <soap:Body>
        <soap:Fault>
          <soap:Code>
            <soap:Value>soap:Receiver</soap:Value>
          </soap:Code>
          <soap:Reason>
            <soap:Text>system failure: getDirectContext</soap:Text>
          </soap:Reason>
          <soap:Detail>
            <Error xmlns="urn:zimbra">
              <Code>service.FAILURE</Code>
              <Trace>com.zimbra.common.service.ServiceException: system failure: getDirectContext
    Code:service.FAILURE
            at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:183)
            at com.zimbra.cs.account.ldap.LdapUtil.getDirContext(LdapUtil.java:219)
            at com.zimbra.cs.account.ldap.LdapUtil.getDirContext(LdapUtil.java:203)
            at com.zimbra.cs.account.ldap.LdapProvisioning.getCosByName(LdapProvisioning.java:1749)
            at com.zimbra.cs.account.ldap.LdapProvisioning.get(LdapProvisioning.java:1719)
            at com.zimbra.cs.account.Provisioning.getCOS(Provisioning.java:1539)
            at com.zimbra.cs.account.ldap.LdapProvisioning.makeAccount(LdapProvisioning.java:3339)
            at com.zimbra.cs.account.ldap.LdapProvisioning.getAccountByQuery(LdapProvisioning.java:457)
            at com.zimbra.cs.account.ldap.LdapProvisioning.getAccountByName(LdapProvisioning.java:586)
            at com.zimbra.cs.account.ldap.LdapProvisioning.get(LdapProvisioning.java:503)
            at com.zimbra.cs.account.ldap.LdapProvisioning.get(LdapProvisioning.java:490)
            at com.zimbra.soap.ZimbraSoapContext.&lt;init>(ZimbraSoapContext.java:191)
            at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:143)
            at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:113)
            at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:272)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
            at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:174)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
            at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
            at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
            at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:81)
            at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:132)
            at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
            at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
            at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
            at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
            at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:716)
            at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:406)
            at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
            at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
            at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
            at org.mortbay.jetty.handler.RewriteHandler.handle(RewriteHandler.java:176)
            at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
            at org.mortbay.jetty.Server.handle(Server.java:313)
            at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
            at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:844)
            at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:644)
            at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
            at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
            at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396)
            at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
    Caused by: javax.naming.CommunicationException: mail.myserver.com:389 [Root exception is java.net.UnknownHostException: mail.myserver.com]
            at com.sun.jndi.ldap.Connection.&lt;init>(Connection.java:197)
            at com.sun.jndi.ldap.LdapClient.&lt;init>(LdapClient.java:118)
            at com.sun.jndi.ldap.LdapClientFactory.createPooledConnection(LdapClientFactory.java:46)
            at com.sun.jndi.ldap.pool.Connections.getOrCreateConnection(Connections.java:185)
            at com.sun.jndi.ldap.pool.Connections.get(Connections.java:126)
            at com.sun.jndi.ldap.pool.Pool.getPooledConnection(Pool.java:129)
            at com.sun.jndi.ldap.LdapPoolManager.getLdapClient(LdapPoolManager.java:310)
            at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1572)
            at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2616)
            at com.sun.jndi.ldap.LdapCtx.&lt;init>(LdapCtx.java:287)
            at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:175)
            at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:193)
            at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:136)
            at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:66)
            at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)
            at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:288)
            at javax.naming.InitialContext.init(InitialContext.java:223)
            at javax.naming.ldap.InitialLdapContext.&lt;init>(InitialLdapContext.java:134)
            at com.zimbra.cs.account.ldap.LdapUtil.getDirContext(LdapUtil.java:214)
            ... 39 more
    Caused by: java.net.UnknownHostException: mail.myserver.com
            at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
            at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
            at java.net.Socket.connect(Socket.java:519)
            at sun.reflect.GeneratedMethodAccessor167.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at com.sun.jndi.ldap.Connection.createSocket(Connection.java:336)
            at com.sun.jndi.ldap.Connection.&lt;init>(Connection.java:184)
            ... 57 more
    </Trace>
            </Error>
          </soap:Detail>
        </soap:Fault>
      </soap:Body>
    </soap:Envelope>
    Sometimes when restarting zimbra via zmcontrol, LDAP refuses to start:
    Code:
    Host mail.myserver.com
            Starting ldap...Done.
    FAILED
    Failed to start slapd.  Attempting debug start to determine error.
    daemon: getaddrinfo() failed: Name or service not known
    After multiple restarts, it will be successful.

    Does anybody know what might be causing this error?
    Is there any known method for reinitializing LDAP / SLAPD without affecting zimbra?
    Is there any way to get a deeper level of logging out of LDAP / slapd?

  2. #2
    Vai0l0 is offline Active Member
    Join Date
    Apr 2008
    Location
    Milan
    Posts
    45
    Rep Power
    7

    Default

    Quote Originally Posted by geterdone View Post
    Does anybody know what might be causing this error?
    Is there any known method for reinitializing LDAP / SLAPD without affecting zimbra?
    Is there any way to get a deeper level of logging out of LDAP / slapd?
    Would be helpful here as well, we have the master LDAP crashing continuously.

    restarting the ldap with zmcontrol start
    solve the problem
    nothing in the log that could give a clue
    Ciao
    AleX

  3. #3
    warrior is offline Member
    Join Date
    Feb 2009
    Posts
    12
    Rep Power
    6

    Default

    Did anybody found a way to solve this?
    We're having exact the same error. The ldap remains working, but after a while (2 or 3 minutes) it stops.
    Restarting it manually solves the issue momentarily..
    Thanks.

  4. #4
    uxbod's Avatar
    uxbod is offline Moderator
    Join Date
    Nov 2006
    Location
    UK
    Posts
    8,016
    Rep Power
    24

    Default

    How much memory does the server have ? Is it a bare metal server or virtualised ?

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. upgrading from 5.0.4 to 5.0.5 opensource
    By smoke in forum Installation
    Replies: 4
    Last Post: 10-19-2008, 10:38 AM
  2. [SOLVED] Build Zimbra..a little problem...
    By Abdelmonam Kouka in forum Developers
    Replies: 33
    Last Post: 05-22-2008, 05:10 AM
  3. 3 testing: LDAP: 389 Failed when restore zimbra
    By victorLeong in forum Administrators
    Replies: 15
    Last Post: 05-24-2007, 06:45 AM
  4. 3.0 to 4.5.3 Upgrade failed (mysql error)
    By dealt in forum Installation
    Replies: 35
    Last Post: 03-19-2007, 10:30 PM
  5. Is it started or not
    By kwelipatton in forum Installation
    Replies: 10
    Last Post: 03-28-2006, 11:11 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •