fatal: Queue report unavailable - mail system is down

[zzzzsg]

Hi All

My Zimbra suddenly went down.

When I did a zmcontrol status, it gave me a "Cannot determine services - exiting" message.

Then I did a zmcontrol stop, and while stopping it gave me "antivirus failed, mailbox failed (tomcat not running ...), mta failed" error messages.

After that, I did a restart, and all is well again.

In my zimbra.log, I have this message:

mail postfix/postqueue[2492]: fatal: Queue report unavailable - mail system is down


Thank you very much in anticipation

===

[zimbra@mail ~]$ zmcontrol status
Cannot determine services - exiting

[zimbra@mail ~]$ zmcontrol stop
Host mail.function.com
Stopping antispam...Done
Stopping antivirus...FAILED


Stopping ldap...Done
Stopping logger...Done
Stopping mailbox...

FAILED
Error: tomcat not running (pid 18225), stale pid file?
Killing mysqld with pid 18326 /opt/zimbra/bin/mysql.server: line 58: kill: (18326) - No such process
.............................. gave up waiting!


Stopping mta...FAILED


Stopping snmp...Done
Stopping spell...Done
You have new mail in /var/spool/mail/zimbra

[zimbra@mail ~]$
[zimbra@mail ~]$
[zimbra@mail ~]$ zmcontrol start
Host mail.function.com
Starting ldap...Done.
Starting antispam...Done.
Starting antivirus...Done.
Starting logger...Done.
Starting mailbox...Done.
Starting mta...Done.
Starting snmp...Done.
Starting spell...Done.
[zimbra@mail ~]$

zimbra.log :



Jun 28 04:54:03 mail zimbramon[3788]: 3788:info: 2006-06-28 04:54:01, STATUS: mail.function.com: antispam: Running
Jun 28 04:54:03 mail zimbramon[3788]: 3788:info: 2006-06-28 04:54:01, STATUS: mail.function.com: antivirus: Running
Jun 28 04:54:03 mail zimbramon[3788]: 3788:info: 2006-06-28 04:54:01, STATUS: mail.function.com: ldap: Running
Jun 28 04:54:03 mail zimbramon[3788]: 3788:info: 2006-06-28 04:54:01, STATUS: mail.function.com: logger: Stopped
Jun 28 04:54:03 mail zimbramon[3788]: 3788:info: 2006-06-28 04:54:01, STATUS: mail.function.com: mailbox: Running
Jun 28 04:54:03 mail zimbramon[3788]: 3788:info: 2006-06-28 04:54:01, STATUS: mail.function.com: mta: Running
Jun 28 04:54:03 mail zimbramon[3788]: 3788:info: 2006-06-28 04:54:01, STATUS: mail.function.com: snmp: Running
Jun 28 04:54:03 mail zimbramon[3788]: 3788:info: 2006-06-28 04:54:01, STATUS: mail.function.com: spell: Running
Jun 28 05:02:04 mail zimbramon[2273]: 2273:info: 2006-06-28 05:02:01, STATUS: : Cannot: determine
Jun 28 05:04:03 mail zimbramon[2314]: 2314:info: 2006-06-28 05:04:01, STATUS: : Cannot: determine
Jun 28 05:06:02 mail zimbramon[2376]: 2376:info: 2006-06-28 05:06:01, STATUS: : Cannot: determine
Jun 28 05:08:02 mail zimbramon[2418]: 2418:info: 2006-06-28 05:08:01, STATUS: : Cannot: determine
Jun 28 12:10:02 mail postfix/postqueue[2492]: fatal: Queue report unavailable - mail system is down
Jun 28 05:10:03 mail zimbramon[2468]: 2468:info: 2006-06-28 05:10:02, DISK: mail.function.com: dev: /dev/sda3, mp: /, tot: 222971, avail: 203258
Jun 28 05:10:03 mail zimbramon[2468]: 2468:info: 2006-06-28 05:10:02, DISK: mail.function.com: dev: /dev/sda1, mp: /boot, tot: 99, avail: 81
Jun 28 05:10:03 mail zimbramon[2469]: 2469:info: 2006-06-28 05:10:02, QUEUE: 0 0
Jun 28 05:10:04 mail zimbramon[2467]: 2467:info: 2006-06-28 05:10:02, STATUS: : Cannot: determine
Jun 28 05:12:02 mail zimbramon[2566]: 2566:info: 2006-06-28 05:12:01, STATUS: : Cannot: determine
Jun 28 05:14:03 mail zimbramon[2606]: 2606:info: 2006-06-28 05:14:01, STATUS: : Cannot: determine
Jun 28 05:16:02 mail zimbramon[2647]: 2647:info: 2006-06-28 05:16:01, STATUS: : Cannot: determine
Jun 28 05:18:02 mail zimbramon[2687]: 2687:info: 2006-06-28 05:18:01, STATUS: : Cannot: determine
Jun 28 12:20:02 mail postfix/postqueue[2743]: fatal: Queue report unavailable - mail system is down
Jun 28 05:20:02 mail zimbramon[2732]: 2732:info: 2006-06-28 05:20:02, DISK: mail.function.com: dev: /dev/sda3, mp: /, tot: 222971, avail: 203258
Jun 28 05:20:02 mail zimbramon[2732]: 2732:info: 2006-06-28 05:20:02, DISK: mail.function.com: dev: /dev/sda1, mp: /boot, tot: 99, avail: 81
Jun 28 05:20:03 mail zimbramon[2729]: 2729:info: 2006-06-28 05:20:02, QUEUE: 0 0
Jun 28 05:20:04 mail zimbramon[2734]: 2734:info: 2006-06-28 05:20:02, STATUS: : Cannot: determine
Jun 28 05:22:02 mail zimbramon[2823]: 2823:info: 2006-06-28 05:22:01, STATUS: : Cannot: determine
Jun 28 05:24:02 mail zimbramon[2863]: 2863:info: 2006-06-28 05:24:01, STATUS: : Cannot: determine
Jun 28 05:26:03 mail zimbramon[2904]: 2904:info: 2006-06-28 05:26:01, STATUS: : Cannot: determine

[phoenix]

What's in the catalina.out log file? Does a reboot or restart of the services work?

[zzzzsg]

No, I cannot find anything in catalina.out.

It was not just the web interface not running.
When I used Thunderbird to go the server, the server also didn't respond.
The imap was obviously not running.

I did a zmcontrol stop and zmcontrol start and everything is okay now.

Trying to figure out when the mail service went down in the first place.

Originally Posted by phoenix

What's in the catalina.out log file? Does a reboot or restart of the services work?

[phoenix]

You'll only find something in catalina.out when tomcat has failed for some reason, normally there's just a few entries in there about startup. Have a look in that log file if it goes down again.

[zzzzsg]

Originally Posted by phoenix

You'll only find something in catalina.out when tomcat has failed for some reason, normally there's just a few entries in there about startup. Have a look in that log file if it goes down again.

This is the catalina.out but it only shows what happened after I re-started zimbra. I could not find any old logs. All the catalina.out.xxxxxx.gz files are unempty.

Thanks.


[root@mail logs]# more catalina.out
Jun 28, 2006 8:23:36 AM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-80
Jun 28, 2006 8:23:37 AM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-443
Jun 28, 2006 8:23:37 AM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-7071
Jun 28, 2006 8:23:37 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 2539 ms
Jun 28, 2006 8:23:37 AM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Jun 28, 2006 8:23:37 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.15
Jun 28, 2006 8:23:38 AM org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
Jun 28, 2006 8:23:38 AM org.apache.catalina.startup.HostConfig deployDescriptor
WARNING: A docBase /opt/zimbra/apache-tomcat-5.5.15/webapps/zimbra inside the host appBase has been specified, and will be igno
red
log4j:WARN No appenders could be found for logger (org.apache.catalina.session.ManagerBase).
log4j:WARN Please initialize the log4j system properly.
Zimbra server reserving server socket port=110 bindaddr=null ssl=false
Zimbra server reserving server socket port=995 bindaddr=null ssl=true
Zimbra server reserving server socket port=143 bindaddr=null ssl=false
Zimbra server reserving server socket port=993 bindaddr=null ssl=true
Zimbra server process is running as root, changing to user=zimbra uid=501 gid=501
Zimbra server process, after change, is running with uid=501 euid=501 gid=501 egid=501
Jun 28, 2006 8:23:42 AM org.apache.coyote.http11.Http11BaseProtocol start
INFO: Starting Coyote HTTP/1.1 on http-80
Jun 28, 2006 8:23:43 AM org.apache.coyote.http11.Http11BaseProtocol start
INFO: Starting Coyote HTTP/1.1 on http-443
Jun 28, 2006 8:23:43 AM org.apache.coyote.http11.Http11BaseProtocol start
INFO: Starting Coyote HTTP/1.1 on http-7071
Jun 28, 2006 8:23:43 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 5775 ms
[root@mail logs]#

[phoenix]

Yes, that's just the normal output. To get any useful info on the failure you'll need to look at that log before you restart tomcat.

[zzzzsg]

Yes, I know. But all the other catalina.out.xxxxx.gz files were emptyp when I gunzipped them. The files are all only 46bytes big - and so there is probably nothing in them.

Should i look at the stacktrace files?



-rw-r--r-- 1 501 501 46 Jun 22 02:00 catalina.out.200606220200.gz
-rw-r--r-- 1 501 501 46 Jun 23 02:00 catalina.out.200606230200.gz
-rw-r--r-- 1 501 501 46 Jun 24 02:00 catalina.out.200606240200.gz
-rw-r--r-- 1 501 501 46 Jun 25 02:00 catalina.out.200606250200.gz
-rw-r--r-- 1 501 501 46 Jun 26 02:00 catalina.out.200606260200.gz
-rw-r--r-- 1 501 501 46 Jun 27 02:00 catalina.out.200606270200.gz
-rw-r--r-- 1 501 501 0 Jun 28 02:00 catalina.out.200606280200

Originally Posted by phoenix

Yes, that's just the normal output. To get any useful info on the failure you'll need to look at that log before you restart tomcat.

[jholder]

Hi,
I was looking at your zimbra log, and the logger was down at the start of the paste of the log.

Do you keep the logger shut off?

[zzzzsg]

Originally Posted by wannabetenor

Hi,
I was looking at your zimbra log, and the logger was down at the start of the paste of the log.

Do you keep the logger shut off?

Hi wannabetenor

The logger has a habit of stopping itself.
I don't know why it was stopped.
That's why there is nothing in catalina.out?

Thank you.

[jholder]

I'm not sure the two are quite unrelated.

Can you post a complete log?

John