Page 2 of 3 FirstFirst 123 LastLast
Results 11 to 20 of 22

Thread: Every night one of the services stops, zimbra stops working.

  1. #11
    lowlow is offline Senior Member
    Join Date
    Mar 2009
    Posts
    50
    Rep Power
    6

    Default

    Thanks for your alertness..
    When I search for errors they are all the same, occurring like 100 times (the last 8 days). I fact I did a search on 'exception'
    It seems to be MySQL related

    Code:
    	Line 1916: 	at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:248)
    	Line 1916: 	at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:248)
    	Line 1957: Caused by: com.mysql.jdbc.exceptions.MySQLNonTransientConnectionException: Server connection failure during transaction. Due to underlying exception: 'java.net.ConnectException: Connection refused'.
    	Line 1957: Caused by: com.mysql.jdbc.exceptions.MySQLNonTransientConnectionException: Server connection failure during transaction. Due to underlying exception: 'java.net.ConnectException: Connection refused'.
    	Line 1957: Caused by: com.mysql.jdbc.exceptions.MySQLNonTransientConnectionException: Server connection failure during transaction. Due to underlying exception: 'java.net.ConnectException: Connection refused'.
    	Line 1957: Caused by: com.mysql.jdbc.exceptions.MySQLNonTransientConnectionException: Server connection failure during transaction. Due to underlying exception: 'java.net.ConnectException: Connection refused'.
    	Line 1959: ** BEGIN NESTED EXCEPTION ** 
    	Line 1961: java.net.ConnectException
    	Line 1966: java.net.ConnectException: Connection refused
    	Line 2030: ** END NESTED EXCEPTION **
    	Line 2034: 	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:888)
    	Line 2048: com.zimbra.common.service.ServiceException: system failure: getting database connection
    	Line 2049: ExceptionId:main:1283158184326:7b538b216ece124f

  2. #12
    waltholman is offline Intermediate Member
    Join Date
    May 2009
    Location
    Tennessee
    Posts
    21
    Rep Power
    5

    Default

    It does look like the mysql daemon is not accepting connections. Maybe it died. What does /opt/zimbra/log/mysql_error.log show you? The last 20 or so lines should give clues into what's going on - do a "tail -n20 /opt/zimbra/log/mysql_error.log" and post the results.

  3. #13
    lowlow is offline Senior Member
    Join Date
    Mar 2009
    Posts
    50
    Rep Power
    6

    Default

    I took a little more than 20 lines to have a better overview

    Code:
    100901 12:31:04  mysqld started
    100901 12:31:08  InnoDB: Started; log sequence number 1 2295311437
    100901 12:31:08 [Note] /opt/zimbra/mysql/libexec/mysqld: ready for connections.
    Version: '5.0.87-log'  socket: '/opt/zimbra/db/mysql.sock'  port: 7306  Source distribution
    100901 14:07:12 [Note] /opt/zimbra/mysql/libexec/mysqld: Normal shutdown
    
    100901 14:07:12  InnoDB: Starting shutdown...
    100901 14:07:14  InnoDB: Shutdown completed; log sequence number 1 2296644922
    100901 14:07:14 [Note] /opt/zimbra/mysql/libexec/mysqld: Shutdown complete
    
    100901 14:07:14  mysqld ended
    
    100901 14:09:18  mysqld started
    100901 14:09:21  InnoDB: Started; log sequence number 1 2296644922
    100901 14:09:21 [Note] /opt/zimbra/mysql/libexec/mysqld: ready for connections.
    Version: '5.0.87-log'  socket: '/opt/zimbra/db/mysql.sock'  port: 7306  Source distribution
    100902  4:35:12 [ERROR] Error writing file '/opt/zimbra/log/myslow.log' (errno: 1)
    100902 11:50:08 [Note] /opt/zimbra/mysql/libexec/mysqld: Normal shutdown
    
    100902 11:50:08  InnoDB: Starting shutdown...
    100902 11:50:10  InnoDB: Shutdown completed; log sequence number 1 2309448341
    100902 11:50:10 [Note] /opt/zimbra/mysql/libexec/mysqld: Shutdown complete
    
    100902 11:50:10  mysqld ended
    
    100902 11:51:21  mysqld started
    100902 11:51:25  InnoDB: Started; log sequence number 1 2309448341
    100902 11:51:25 [Note] /opt/zimbra/mysql/libexec/mysqld: ready for connections.
    Version: '5.0.87-log'  socket: '/opt/zimbra/db/mysql.sock'  port: 7306  Source distribution
    100903 12:02:23 [Note] /opt/zimbra/mysql/libexec/mysqld: Normal shutdown
    
    100903 12:02:23  InnoDB: Starting shutdown...
    100903 12:02:25  InnoDB: Shutdown completed; log sequence number 1 2315138450
    100903 12:02:25 [Note] /opt/zimbra/mysql/libexec/mysqld: Shutdown complete
    
    100903 12:02:25  mysqld ended
    
    100903 12:04:34  mysqld started
    100903 12:04:38  InnoDB: Started; log sequence number 1 2315138450
    100903 12:04:38 [Note] /opt/zimbra/mysql/libexec/mysqld: ready for connections.
    Version: '5.0.87-log'  socket: '/opt/zimbra/db/mysql.sock'  port: 7306  Source distribution
    100906 11:37:59 [Note] /opt/zimbra/mysql/libexec/mysqld: Normal shutdown
    
    100906 11:37:59  InnoDB: Starting shutdown...
    100906 11:38:05  InnoDB: Shutdown completed; log sequence number 1 2579486918
    100906 11:38:05 [Note] /opt/zimbra/mysql/libexec/mysqld: Shutdown complete
    
    100906 11:38:06  mysqld ended
    
    100906 11:39:16  mysqld started
    100906 11:39:19  InnoDB: Started; log sequence number 1 2579486918
    100906 11:39:19 [Note] /opt/zimbra/mysql/libexec/mysqld: ready for connections.
    Version: '5.0.87-log'  socket: '/opt/zimbra/db/mysql.sock'  port: 7306  Source distribution

  4. #14
    waltholman is offline Intermediate Member
    Join Date
    May 2009
    Location
    Tennessee
    Posts
    21
    Rep Power
    5

    Default

    I see where it failed to write to the slow query log file. What is the free space on your zimbra install? Also, I assume that shutdown & startups are from a backup script? Each of the restarts appear normal, so if they were happening due to a zimbra restart, I don't understand where the connection refused was coming from.
    Have you ran the zmfixperms script to make sure permissions are ok (/opt/zimbra/libexec/zmfixperms -extended)?

  5. #15
    lowlow is offline Senior Member
    Join Date
    Mar 2009
    Posts
    50
    Rep Power
    6

    Default

    Thanks again for your feedback,
    Indeed I had a restart script that runs at 5am every day, since I've had the problems with the zimbra.

    I ran the fixperms script, thanks for the tip, and disabled the restart script. I'll try it without it for a while.

    Concerning the disk space:

    Code:
    zimbramail:/# df -a
    Filesystem           1K-blocks      Used Available Use% Mounted on
    tmpfs                  1681872         0   1681872   0% /lib/init/rw
    proc                         0         0         0   -  /proc
    sysfs                        0         0         0   -  /sys
    tmpfs                  1681872         0   1681872   0% /dev/shm
    devpts                       0         0         0   -  /dev/pts
    simfs                151187140 107137888  36369384  75% /
    I believe it's normal some values are 0% free


    Edit: tonight, same thing

    Code:
    zimbra@zimbramail:/$ zmcontrol status
    Host zimbramail.gent.imm.be
            ldap                    Running
            logger                  Running
            mailbox                 Stopped
                    zmmailboxdctl is not running.
            memcached               Running
            mta                     Running
            snmp                    Running
            spell                   Running
            stats                   Running
    Can I post some log file?
    Last edited by lowlow; 09-08-2010 at 12:51 AM.

  6. #16
    waltholman is offline Intermediate Member
    Join Date
    May 2009
    Location
    Tennessee
    Posts
    21
    Rep Power
    5

    Default

    Not sure what else is going on. From your earlier posting, the mysql connection refused errors indicate that it's not running. Next time you have a problem, do a 'ps ax | grep mysql' and see if the daemon is still running. If it's not running, you need to figure out what's killing it. I've seen corrupted databases cause fatal exceptions to mysql that will cause it to abort...

    Is there any time correlation to the outages? ie: Does it seem to shutdown after a certain number of hours uptime etc...? Which exact version of ZCS are you running (zmcontrol -v)?

  7. #17
    waltholman is offline Intermediate Member
    Join Date
    May 2009
    Location
    Tennessee
    Posts
    21
    Rep Power
    5

    Default

    Also, in the /var/log/messages file you attached in your first post, the condition already existed at the start of the log file with no indication of what occurred prior to it. Can you attach a log file that would include the items leading up to the failure?

  8. #18
    lowlow is offline Senior Member
    Join Date
    Mar 2009
    Posts
    50
    Rep Power
    6

    Default

    Code:
    zimbra@zimbramail:/$ ps ax | grep mysql
     4994 ?        S      0:00 /bin/sh /opt/zimbra/mysql/bin/mysqld_safe --defaults-file=/opt/zimbra/conf/my.cnf --external-locking --log-error=/opt/zimbra/log/mysql_error.log --ledir=/opt/zimbra/mysql/libexec
     5099 ?        Sl     1:35 /opt/zimbra/mysql/libexec/mysqld --defaults-file=/opt/zimbra/conf/my.cnf --basedir=/opt/zimbra/mysql --datadir=/opt/zimbra/db/data --pid-file=/opt/zimbra/db/mysql.pid --skip-external-locking --port=7306 --socket=/opt/zimbra/db/mysql.sock --external-locking
     9393 ?        S      0:13 /usr/bin/perl -w? /opt/zimbra/libexec/zmstat-mysql
    26411 pts/0    S+     0:00 grep mysql
    In attachment I added the older /var/log/messages files, next to the latest, and a bunch of other files too, like /opt/zimbra/zmmailboxd.out.2010090... because it is the zmmailboxd that is always crashing lately.

    Code:
    zimbra@zimbramail:/$ zmcontrol -v
    
    Release 6.0.5_GA_2213.DEBIAN5 DEBIAN5 FOSS edition.
    Thanks in advance!
    Attached Files Attached Files

  9. #19
    waltholman is offline Intermediate Member
    Join Date
    May 2009
    Location
    Tennessee
    Posts
    21
    Rep Power
    5

    Default

    Is there any reason you've not updated ZCS to the most recent version? I don't recall there being any specific bugs relating to your problem, but you might see if upgrading to zcs-6.0.8 + patch resolves this.

  10. #20
    lowlow is offline Senior Member
    Join Date
    Mar 2009
    Posts
    50
    Rep Power
    6

    Default

    Quote Originally Posted by waltholman View Post
    Is there any reason you've not updated ZCS to the most recent version? I don't recall there being any specific bugs relating to your problem, but you might see if upgrading to zcs-6.0.8 + patch resolves this.
    Weirdly enough the zimbra mailserver was working for almost 3 days straight. Today the same error as before, no surprise.

    Sorry for the rookie question, but how can I do the upgrade without getting prompted to re-enter all the information that I gave on the first install.
    I googled and searched the forum for this question but the only instructions I found were: unpack the installation package and run the install.sh
    Is this without the prompting all the time?
    Thx

Page 2 of 3 FirstFirst 123 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Upgrade to ZCS 5.10
    By blozancic in forum Installation
    Replies: 0
    Last Post: 10-21-2008, 08:03 AM
  2. [SOLVED] parts_decode_ext error
    By jsabater in forum Administrators
    Replies: 7
    Last Post: 10-13-2008, 07:24 AM
  3. Major Issue - 5.0RC2 NE to 5.0GA NE failed
    By DougWare in forum Installation
    Replies: 7
    Last Post: 01-06-2008, 09:56 PM
  4. Unable to start tomcat
    By chanck in forum Administrators
    Replies: 11
    Last Post: 06-11-2006, 12:58 AM
  5. 3.1 on FC4 problems
    By cohnhead in forum Installation
    Replies: 8
    Last Post: 05-26-2006, 11:16 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •