Zimbra is going down

[fernandoflorez]

Hello guys,

I upgraded to the latest zimbra (4.0.3) and the server is going down randomly. This are the messages i'm getting on zimbra.log

Nov  2 09:00:13 studio zimbramon[13456]: 13456:info: 2006-11-02 09:00:01, STATUS: onelx.com: antispam: Running 
Nov  2 09:00:13 studio zimbramon[13456]: 13456:info: 2006-11-02 09:00:01, STATUS: onelx.com: antivirus: Running 
Nov  2 09:00:13 studio zimbramon[13456]: 13456:info: 2006-11-02 09:00:01, STATUS: onelx.com: ldap: Running 
Nov  2 09:00:13 studio zimbramon[13456]: 13456:info: 2006-11-02 09:00:01, STATUS: onelx.com: logger: Running 
Nov  2 09:00:13 studio zimbramon[13456]: 13456:info: 2006-11-02 09:00:01, STATUS: onelx.com: mailbox: Running 
Nov  2 09:00:13 studio zimbramon[13456]: 13456:info: 2006-11-02 09:00:01, STATUS: onelx.com: mta: Running 
Nov  2 09:00:13 studio zimbramon[13456]: 13456:info: 2006-11-02 09:00:01, STATUS: onelx.com: snmp: Running 
Nov  2 09:00:13 studio zimbramon[13456]: 13456:info: 2006-11-02 09:00:01, STATUS: onelx.com: spell: Running 
Nov  2 09:01:25 studio amavis[25748]: Net::Server: 2006/11/02-09:01:25 Server closing!
Nov  2 09:01:28 studio clamd[25779]: Pid file removed. 
Nov  2 09:01:28 studio clamd[25779]: Exiting (clean) 
Nov  2 09:01:28 studio clamd[25779]: --- Stopped at Thu Nov  2 09:01:28 2006 
Nov  2 09:01:29 studio postfix/postfix-script: stopping the Postfix mail system
Nov  2 09:01:29 studio postfix/master[25798]: terminating on signal 15
Nov  2 09:01:29 studio zmtomcatmgr[14146]: threaddump requested
Nov  2 09:01:29 studio zmtomcatmgr[25450]: sending SIQUIT to tomcat/JVM process 25451
Nov  2 09:01:31 studio zmtomcatmgr[14153]: stop requested
Nov  2 09:01:31 studio zmtomcatmgr[14153]: waiting for manager process 25450 to die
Nov  2 09:01:31 studio zmtomcatmgr[25450]: shutdown requested, sending TERM signal to 25451
Nov  2 09:01:34 studio zmtomcatmgr[25450]: tomcat/JVM process exited (waitpid expected 25451 got 25451)
Nov  2 09:01:34 studio zmtomcatmgr[25450]: manager woke up from wait on tomcat/JVM with pid 25451
Nov  2 09:01:35 studio zmtomcatmgr[14153]: manager process 25450 died, shutdown completed
Nov  2 09:02:08 studio zmtomcatmgr[14315]: status requested
Nov  2 09:02:08 studio zmtomcatmgr[14315]: file /opt/zimbra/log/zmtomcatmgr.pid does not exist
Nov  2 09:02:08 studio zmtomcatmgr[14315]: assuming no other instance is running
Nov  2 09:02:08 studio zmtomcatmgr[14315]: no manager process is running
Nov  2 14:02:08 studio postfix/postqueue[14325]: fatal: Queue report unavailable - mail system is down
Nov  2 09:02:09 studio zimbramon[14160]: 14160:info: 2006-11-02 09:02:02, STATUS: onelx.com: antispam: Stopped 
Nov  2 09:02:09 studio zimbramon[14160]: 14160:info: 2006-11-02 09:02:02, STATUS: onelx.com: antivirus: Stopped 
Nov  2 09:02:09 studio zimbramon[14160]: 14160:info: 2006-11-02 09:02:02, STATUS: onelx.com: ldap: Running 
Nov  2 09:02:09 studio zimbramon[14160]: 14160:info: 2006-11-02 09:02:02, STATUS: onelx.com: logger: Running 
Nov  2 09:02:09 studio zimbramon[14160]: 14160:info: 2006-11-02 09:02:02, STATUS: onelx.com: mailbox: Stopped 
Nov  2 09:02:09 studio zimbramon[14160]: 14160:info: 2006-11-02 09:02:02, STATUS: onelx.com: mta: Stopped 
Nov  2 09:02:09 studio zimbramon[14160]: 14160:info: 2006-11-02 09:02:02, STATUS: onelx.com: snmp: Running 
Nov  2 09:02:09 studio zimbramon[14160]: 14160:info: 2006-11-02 09:02:02, STATUS: onelx.com: spell: Running 
Nov  2 09:03:25 studio amavis[14582]: starting.  /opt/zimbra/amavisd/sbin/amavisd at onelx.com amavisd-new-2.4.1 (20060508), Unicode aware, LANG=es_PE.UTF-8
Nov  2 09:03:25 studio amavis[14582]: user=500, EUID: 500 (500);  group=, EGID: 500 501 500 5 (500 501 500 5); log_level=1
Nov  2 09:03:25 studio amavis[14582]: Perl version               5.008005
Nov  2 09:03:26 studio amavis[14582]: SpamControl: init_pre_chroot done
Nov  2 09:03:26 studio amavis[14583]: Net::Server: Process Backgrounded
Nov  2 09:03:26 studio amavis[14583]: Net::Server: 2006/11/02-09:03:26 Amavis (type Net::Server::PreForkSimple) starting! pid(14583)
Nov  2 09:03:26 studio amavis[14583]: Net::Server: Binding to UNIX socket file /opt/zimbra/amavisd/amavisd.sock using SOCK_STREAM
Nov  2 09:03:26 studio amavis[14583]: Net::Server: Binding to TCP port 10024 on host 127.0.0.1
Nov  2 09:03:26 studio amavis[14583]: Net::Server: Group Not Defined.  Defaulting to EGID '500 501 500 5'
Nov  2 09:03:26 studio amavis[14583]: Net::Server: User Not Defined.  Defaulting to EUID '500'
Nov  2 09:03:26 studio amavis[14583]: Module Amavis::Conf        2.065
Nov  2 09:03:26 studio amavis[14583]: Module Archive::Tar        1.29
Nov  2 09:03:26 studio amavis[14583]: Module Archive::Zip        1.16
Nov  2 09:03:26 studio amavis[14583]: Module BerkeleyDB          0.28
Nov  2 09:03:26 studio amavis[14583]: Module Compress::Zlib      1.41
Nov  2 09:03:26 studio amavis[14583]: Module Convert::TNEF       0.17
Nov  2 09:03:26 studio amavis[14583]: Module Convert::UUlib      1.06
Nov  2 09:03:26 studio amavis[14583]: Module DBD::mysql          3.0006
Nov  2 09:03:26 studio amavis[14583]: Module DBI                 1.51
Nov  2 09:03:26 studio amavis[14583]: Module DB_File             1.814
Nov  2 09:03:26 studio amavis[14583]: Module Digest::MD5         2.33
Nov  2 09:03:26 studio amavis[14583]: Module MIME::Entity        5.420
Nov  2 09:03:26 studio amavis[14583]: Module MIME::Parser        5.420
Nov  2 09:03:26 studio amavis[14583]: Module MIME::Tools         5.420
Nov  2 09:03:26 studio amavis[14583]: Module Mail::Header        1.74
Nov  2 09:03:26 studio amavis[14583]: Module Mail::Internet      1.74
Nov  2 09:03:26 studio amavis[14583]: Module Mail::SpamAssassin  3.001003
Nov  2 09:03:26 studio amavis[14583]: Module Net::Cmd            2.26
Nov  2 09:03:26 studio amavis[14583]: Module Net::DNS            0.57
Nov  2 09:03:26 studio amavis[14583]: Module Net::LDAP           0.33
Nov  2 09:03:26 studio amavis[14583]: Module Net::SMTP           2.29
Nov  2 09:03:26 studio amavis[14583]: Module Net::Server         0.93
Nov  2 09:03:26 studio amavis[14583]: Module Time::HiRes         1.87
Nov  2 09:03:26 studio amavis[14583]: Module Unix::Syslog        0.99
Nov  2 09:03:26 studio amavis[14583]: Amavis::DB code      loaded
Nov  2 09:03:26 studio amavis[14583]: Amavis::Cache code   loaded
Nov  2 09:03:26 studio amavis[14583]: SQL base code        NOT loaded
Nov  2 09:03:26 studio amavis[14583]: SQL::Log code        NOT loaded
Nov  2 09:03:26 studio amavis[14583]: SQL::Quarantine      NOT loaded
Nov  2 09:03:26 studio amavis[14583]: Lookup::SQL code     NOT loaded
Nov  2 09:03:26 studio amavis[14583]: Lookup::LDAP code    loaded
Nov  2 09:03:26 studio amavis[14583]: AM.PDP-in proto code loaded
Nov  2 09:03:26 studio amavis[14583]: SMTP-in proto code   loaded
Nov  2 09:03:26 studio amavis[14583]: Courier proto code   NOT loaded
Nov  2 09:03:26 studio amavis[14583]: SMTP-out proto code  loaded
Nov  2 09:03:26 studio amavis[14583]: Pipe-out proto code  NOT loaded
Nov  2 09:03:26 studio amavis[14583]: BSMTP-out proto code NOT loaded
Nov  2 09:03:26 studio amavis[14583]: Local-out proto code loaded
Nov  2 09:03:26 studio amavis[14583]: OS_Fingerprint code  NOT loaded
Nov  2 09:03:26 studio amavis[14583]: ANTI-VIRUS code      loaded
Nov  2 09:03:26 studio amavis[14583]: ANTI-SPAM code       loaded
Nov  2 09:03:26 studio amavis[14583]: ANTI-SPAM-SA code    loaded
Nov  2 09:03:26 studio amavis[14583]: Unpackers code       loaded
Nov  2 09:03:26 studio amavis[14583]: Found $file            at /usr/bin/file
Nov  2 09:03:26 studio amavis[14583]: Found $dspam           at /opt/zimbra/dspam/bin/dspam
Nov  2 09:03:26 studio amavis[14583]: Internal decoder for .mail
Nov  2 09:03:26 studio amavis[14583]: Internal decoder for .asc 
Nov  2 09:03:26 studio amavis[14583]: Internal decoder for .uue 
Nov  2 09:03:26 studio amavis[14583]: Internal decoder for .hqx 
Nov  2 09:03:26 studio amavis[14583]: Internal decoder for .ync 
Nov  2 09:03:26 studio amavis[14583]: No decoder for       .F    tried: unfreeze, freeze -d, melt, fcat
Nov  2 09:03:26 studio amavis[14583]: Found decoder for    .Z    at /usr/bin/gzip -d
Nov  2 09:03:26 studio amavis[14583]: Internal decoder for .gz  
Nov  2 09:03:26 studio amavis[14583]: Found decoder for    .bz2  at /usr/bin/bzip2 -d
Nov  2 09:03:26 studio amavis[14583]: No decoder for       .lzo  tried: lzop -d
Nov  2 09:03:26 studio amavis[14583]: Found decoder for    .rpm  at /usr/bin/rpm2cpio
Nov  2 09:03:26 studio amavis[14583]: Found decoder for    .cpio at /usr/bin/pax
Nov  2 09:03:26 studio amavis[14583]: Found decoder for    .tar  at /usr/bin/pax
Nov  2 09:03:26 studio amavis[14583]: Found decoder for    .deb  at /usr/bin/ar
Nov  2 09:03:26 studio amavis[14583]: Internal decoder for .zip 
Nov  2 09:03:26 studio amavis[14583]: No decoder for       .rar  tried: rar, unrar
Nov  2 09:03:26 studio amavis[14583]: No decoder for       .arj  tried: arj, unarj
Nov  2 09:03:26 studio amavis[14583]: No decoder for       .arc  tried: nomarch, arc
Nov  2 09:03:26 studio amavis[14583]: No decoder for       .zoo  tried: zoo
Nov  2 09:03:26 studio amavis[14583]: Found decoder for    .lha  at /usr/bin/lha
Nov  2 09:03:26 studio amavis[14583]: No decoder for       .cab  tried: cabextract
Nov  2 09:03:26 studio amavis[14583]: No decoder for       .tnef tried: tnef
Nov  2 09:03:26 studio amavis[14583]: Internal decoder for .tnef
Nov  2 09:03:26 studio amavis[14583]: Found decoder for    .exe  at /usr/bin/lha
Nov  2 09:03:26 studio amavis[14583]: Using internal av scanner code for (primary) ClamAV-clamd
Nov  2 09:03:26 studio clamd[14631]: Daemon started. 
Nov  2 09:03:26 studio clamd[14631]: clamd daemon 0.88.4 (OS: linux-gnu, ARCH: i386, CPU: i686) 
Nov  2 09:03:26 studio clamd[14631]: Log file size limited to 20971520 bytes. 
Nov  2 09:03:27 studio amavis[14583]: Creating db in /opt/zimbra/amavisd/db/; BerkeleyDB 0.28, libdb 4.2
Nov  2 09:03:27 studio clamd[14631]: Reading databases from /opt/zimbra/clamav/db 
Nov  2 09:03:27 studio amavis[14583]: SpamControl: initializing Mail::SpamAssassin
Nov  2 09:03:28 studio postfix/postfix-script: fatal: the Postfix mail system is not running
Nov  2 09:03:28 studio zmtomcatmgr[14737]: threaddump requested
Nov  2 09:03:28 studio zmtomcatmgr[14737]: file /opt/zimbra/log/zmtomcatmgr.pid does not exist
Nov  2 09:03:28 studio zmtomcatmgr[14737]: assuming no other instance is running
Nov  2 09:03:28 studio zmtomcatmgr[14737]: no manager process is running
Nov  2 09:03:29 studio clamd[14631]: Protecting against 75255 viruses. 
Nov  2 09:03:29 studio amavis[14583]: SpamControl: init_pre_fork done
Nov  2 09:03:30 studio zmtomcatmgr[14755]: stop requested
Nov  2 09:03:30 studio zmtomcatmgr[14755]: file /opt/zimbra/log/zmtomcatmgr.pid does not exist
Nov  2 09:03:30 studio zmtomcatmgr[14755]: assuming no other instance is running
Nov  2 09:03:30 studio zmtomcatmgr[14755]: no manager process is running
Nov  2 09:04:13 studio zmtomcatmgr[14912]: status requested
Nov  2 09:04:13 studio zmtomcatmgr[14912]: file /opt/zimbra/log/zmtomcatmgr.pid does not exist
Nov  2 09:04:13 studio zmtomcatmgr[14912]: assuming no other instance is running
Nov  2 09:04:13 studio zmtomcatmgr[14912]: no manager process is running

Hope you guys can help.

Thanks,
Fernando

[fernandoflorez]

Any ideas?

[fernandoflorez]

Just to let you guys know that i downgrade to zimbra 4.0.2 and everything is working smooth again

Thanks!
Fernando

[jholder]

Yikes proto,
I'd recommend you do a backup right away.
Downgrades are never recommended, but it shouldn't cause you any trouble.

[fernandoflorez]

Ya, i already did a backup. It's really strange i had this problem... Don't know the cause of it yet.

Thanks,
Fernando

[gaspode]

We are having the same problem with our installation and are currently looking in to it with the Zimbra support. I'll post here if we get a workable solution.

[Phantasmagoria]

Here have same problem too...installed on 2 Debian system, one is clean install, both have same problem.

[fernandoflorez]

Thanks! Plz do let us know

Thanks,
Fernando

[gaspode]

We might have something of a solution. We had problems with "zmprov help config" complaining of an untrusted server which was solved by importing the CA certificate manually

keytool -import -alias (hostname)_root -keystore /opt/zimbra/java/jre/lib/security/cacerts -storepass changeit
-trustcacerts -file CERT_FILE

This resulted in "zmprov help config" complaining about a failed authentication (ERROR: account.AUTH_FAILED (authentication failed for zimbra)) which was solved by resetting the LDAP password by zmldappasswd.

Between these steps we restarted the tomcat to make sure the changes were acted upon. This far our server hasn't gone down, so hopefully it took care of it.