Results 1 to 10 of 10

Thread: Zimbra Dying - What is the best way to debug this?

  1. #1
    Jazzhands is offline Active Member
    Join Date
    May 2006
    Posts
    33
    Rep Power
    9

    Default Zimbra Dying - What is the best way to debug this?

    I installed Zimbra on RHEL4 and it went pretty smooth. I had previously installed open xchange. I did my best to uninstall that beast but there was so much I had to do to get that working I can't even be sure I completely cleaned it up. That said, zimbra installed without much issue, the mail server ran for a few days, everything was great. I was able send/receive mail, setup new accounts, etc.

    One morning I noticed I got an error that I "could not connect to mail server". Hmm, I logged onto the box, ran a zmcontrol status and everything looked ok. I stopped/started zimbra and boom, everything is working again. This started to happen on a daily basis.

    This is not production yet so I am just now coming back to this after a few days. I started to debug it but realized there are like 15-20 different log files, and I really have no error to search for. The mail server stops responding and I need to restart. That's all the info I have.

    I could not convince my client that running the web server and mail server on the same box was not a good idea, so perhaps the other apache2 install is causing problems? I made sure to remove tomcat before installing zimbra but that could be in issue

    What I am really looking for in this initial post is information on how to debug. There are a lot of log files and I dont' even know when the server exactly shuts down, so finding the actual error seems daunting. Can anyone point me in the write place to start looking for the problem? I'd really appreciate it!

  2. #2
    jholder's Avatar
    jholder is offline Former Zimbran
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    20

    Default

    Hi Jazzhands,

    Are you a piano player? I love jazz on the piano!

    Anywo, let's start with the log. It can often give a glimpse into the world of your server.

    It can be found in /var/log/zimbra.log

    You might as well post it as an attachment rather than just copying and pasting it.

    Good luck,
    jh

  3. #3
    Jazzhands is offline Active Member
    Join Date
    May 2006
    Posts
    33
    Rep Power
    9

    Default

    Ah, haha no. It would be a cool nick for someone like that tho, and I do love jazz as well. It is an obscure reference to a Baltimore Ravens player, Ray Lewis. I am a big Steeler fan and we hate the Ravens. Many Steeler fans mock Ray ray with this nickname. I sorta dig it and even named my mage in World of Warcraft this. Wow did I ever just peg myself as the uber dork. Anyway...

    I compiled the last 4 days log files. I know this is a lot but I'm assuming you pros might have things you can ctrl-f/scan for that I don't know about... at least I hope so. Please let me know if you need anything else, I . Thank you!!!

    Doh, file too large to attach. I'll host it and take it down when this is resolved:
    ***DELETED -- SEE UPDATED FILE BELOW***
    Last edited by Jazzhands; 05-26-2006 at 06:21 AM.

  4. #4
    bobby is offline Zimbra Employee
    Join Date
    Nov 2005
    Posts
    518
    Rep Power
    10

    Default

    check "zmcontrol status" first to verify whether everything is running. then look for errors in the logs. the two main logs are

    /var/log/zimbra.log (postfix; mail delivery)
    /opt/zimbra/log/zimbra.log (tomcat; user access)

  5. #5
    Jazzhands is offline Active Member
    Join Date
    May 2006
    Posts
    33
    Rep Power
    9

    Default

    The server stopped responding within the last hour or two. This allowed me to have a smaller sample of log files to work with. I found this section as the only section not acting normal:

    2006-05-25 01:30:06,958 INFO [http-7071-Processor50] [ip=127.0.0.1;] SoapEngine - handler exception
    com.zimbra.cs.account.AccountServiceException: authentication failed for zimbra (empty password)
    at com.zimbra.cs.account.AccountServiceException.AUTH _FAILED(AccountServiceException.java:77)
    at com.zimbra.cs.account.ldap.LdapProvisioning.authAc count(LdapProvisioning.java:2065)
    at com.zimbra.cs.service.admin.Auth.handle(Auth.java: 82)
    at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEng ine.java:255)
    at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:163)
    at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:84)
    at com.zimbra.soap.SoapServlet.doPost(SoapServlet.jav a:228)
    at javax.servlet.http.HttpServlet.service(HttpServlet .java:709)
    at com.zimbra.cs.servlet.ZimbraServlet.service(Zimbra Servlet.java:154)
    at javax.servlet.http.HttpServlet.service(HttpServlet .java:802)
    at org.apache.catalina.core.ApplicationFilterChain.in ternalDoFilter(ApplicationFilterChain.java:252)
    at org.apache.catalina.core.ApplicationFilterChain.do Filter(ApplicationFilterChain.java:173)
    at org.apache.catalina.core.StandardWrapperValve.invo ke(StandardWrapperValve.java:213)
    at org.apache.catalina.core.StandardContextValve.invo ke(StandardContextValve.java:178)
    at org.apache.catalina.core.StandardHostValve.invoke( StandardHostValve.java:126)
    at org.apache.catalina.valves.ErrorReportValve.invoke (ErrorReportValve.java:105)
    at org.apache.catalina.core.StandardEngineValve.invok e(StandardEngineValve.java:107)
    at org.apache.catalina.valves.AccessLogValve.invoke(A ccessLogValve.java:541)
    at org.apache.catalina.connector.CoyoteAdapter.servic e(CoyoteAdapter.java:148)
    at org.apache.coyote.http11.Http11Processor.process(H ttp11Processor.java:869)
    at org.apache.coyote.http11.Http11BaseProtocol$Http11 ConnectionHandler.processConnection(Http11BaseProt ocol.java:667)
    at org.apache.tomcat.util.net.PoolTcpEndpoint.process Socket(PoolTcpEndpoint.java:527)
    at org.apache.tomcat.util.net.LeaderFollowerWorkerThr ead.runIt(LeaderFollowerWorkerThread.java:80)
    at org.apache.tomcat.util.threads.ThreadPool$ControlR unnable.run(ThreadPool.java:684)
    at java.lang.Thread.run(Thread.java:595)


    I read another thread (zmtrainsa exception) here that had a similar error but for a different reason. My server simply dies at random times. Well it doesn't really die, it just stops accepting connections. Running zmcontrol status shows all systems normal. I reset the LDAP password but I'm unsure if that will work. The major problem is that I can restart zimbra and it will start working again, but if I implement a fix it might be a few days before I know if it even worked. I dont' know of a way to reproduce this issue.

  6. #6
    jholder's Avatar
    jholder is offline Former Zimbran
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    20

    Default

    Do you have any more? I'm looking for a "caused by" line.

  7. #7
    Jazzhands is offline Active Member
    Join Date
    May 2006
    Posts
    33
    Rep Power
    9

    Default

    Ok, the orignal file is now deleted. Here is an updated log. I did find some definite errors by searching for the "Caused" line. Thank you for that, I was going blind reading 1000s of lines of logs without some text to key on. Here is the updated file:

    http://www.evalonline.com/zimbratmp/zimbra_log.txt

  8. #8
    KevinH's Avatar
    KevinH is offline Expert Member
    Join Date
    Aug 2005
    Location
    San Mateo, CA
    Posts
    4,789
    Rep Power
    19

    Default

    The only major error I see in that log is MySQL is not running or Tomcat can't connect to it. Can you verify that MySQL is running? Maybe try to restart it.
    Looking for new beta users -> Co-Founder of Acompli. Previously worked at Zimbra (and Yahoo! & VMware) since 2005.

  9. #9
    Jazzhands is offline Active Member
    Join Date
    May 2006
    Posts
    33
    Rep Power
    9

    Default

    Yes, everything is running. I thought I had found the issue. The message I get from Thunderbird is "The connection to the server timed out." This made me go into the server and up the max threads in IMAP to 1000. This did not help, this morning, once again, the server has stopped responding. And as usual zomcontrol status shows everything running just fine. I am completely lost on this issue.

  10. #10
    KevinH's Avatar
    KevinH is offline Expert Member
    Join Date
    Aug 2005
    Location
    San Mateo, CA
    Posts
    4,789
    Rep Power
    19

    Default

    does "ps -ef | grep tomcat" show tomcat is running? If so you can try to get a stack dump:

    kill -QUIT
    Looking for new beta users -> Co-Founder of Acompli. Previously worked at Zimbra (and Yahoo! & VMware) since 2005.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. zmtlsctl give LDAP error
    By sourcehound in forum Administrators
    Replies: 5
    Last Post: 03-11-2007, 03:48 PM
  2. Replies: 7
    Last Post: 01-24-2007, 11:03 PM
  3. Logger
    By jholder in forum Installation
    Replies: 24
    Last Post: 03-31-2006, 11:50 AM
  4. Fedora Core 3, Clean Install - Not working!
    By pcjackson in forum Installation
    Replies: 17
    Last Post: 03-05-2006, 07:38 PM
  5. Monitoring : Data not yet avalaible
    By s3nz3x in forum Installation
    Replies: 7
    Last Post: 11-30-2005, 07:18 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •