[SOLVED] zmclamdctl is not running

[SamTzu]

Ones again Amavis stopped working without any visible reason.
(No updates/upgrades anything.)

zmcontrol status sais...
zmclamdctl is not running
(Seems there is all ready an instance running?)

I had to go to admin panel and disable Antivirus and manually reque all the mail to get all the queues empy.

I don't want to sound like I'm whining (except I am) but the spam/virus filtering on Zimbra seems to be too unstable.

The "Server Statistics" shows the mail servers Spam_Activity stops dead on 00:00 hours.
(Yet another "maintanance" bug?)

Ubuntu 6.06.1/Zimbra 5.04

[phoenix]

Ones again Amavis stopped working without any visible reason.
(No updates/upgrades anything.)

zmcontrol status sais...
zmclamdctl is not running
(Seems there is all ready an instance running?)

How about some information from the log files? What happens if you try to restart it - does it run OK or fail and if it fails what are the log entries? How much RAM is on this system? Is this a Network Edition or Open Source install?

I had to go to admin panel and disable Antivirus and manually reque all the mail to get all the queues empy.

That is the normal thing to get the mail from the deferred queue.

I don't want to sound like I'm whining (except I am) but the spam/virus filtering on Zimbra seems to be too unstable.

The "Server Statistics" shows the mail servers Spam_Activity stops dead on 00:00 hours.
(Yet another "maintanance" bug?)

Have you searched bugzilla for similar incidents or filed and bug report or even searched the forums for resolutions/troubleshooting tips on a/v or Server Statistics problems?

It's near impossible to help when you give so little useful debugging information.

[SamTzu]

Thanx for replying so fast.

Here are some lines from /var/log/zimbra.log
...
Jul 28 12:41:26 server17 amavis[29793]: (29793-17) ClamAV-clamd: Can't connect to INET socket 127.0.0.1:3310: Connection refused, retrying (1)
Jul 28 12:41:26 server17 amavis[32335]: (32335-12) (!)ClamAV-clamd: Can't connect to INET socket 127.0.0.1:3310: Connection refused, retrying (2)
Jul 28 12:41:26 server17 amavis[32332]: (32332-14) (!)ClamAV-clamd: Can't connect to INET socket 127.0.0.1:3310: Connection refused, retrying (2)
Jul 28 12:41:26 server17 amavis[30976]: (30976-17) (!)ClamAV-clamd: Can't connect to INET socket 127.0.0.1:3310: Connection refused, retrying (2)
Jul 28 12:41:27 server17 amavis[29793]: (29793-17) (!)ClamAV-clamd: Can't connect to INET socket 127.0.0.1:3310: Connection refused, retrying (2)
Jul 28 12:41:31 server17 amavis[927]: (00927-11) (!)run_av (ClamAV-clamd, built-in i/f): Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 75) line 310
.
Jul 28 12:41:31 server17 amavis[927]: (00927-11) (!!)ClamAV-clamd av-scanner FAILED: CODE(0x818e07c) Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval
75) line 310. at (eval 75) line 511.
Jul 28 12:41:31 server17 amavis[927]: (00927-11) (!!)WARN: all primary virus scanners failed, considering backups
Jul 28 12:41:31 server17 amavis[927]: (00927-11) (!!)TROUBLE in check_mail: virus_scan FAILED: virus_scan: ALL VIRUS SCANNERS FAILED: ClamAV-clamd av-scanner FAILED: CODE(0x818e07c) Too many retries to talk to 127.0.0.1
:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 75) line 310. at (eval 75) line 511.
Jul 28 12:41:31 server17 amavis[927]: (00927-11) (!)PRESERVING EVIDENCE in /opt/zimbra/data/amavisd/tmp/amavis-20080728T124124-00927
...

How ever no such file exists.

[phoenix]

.... and the answers to the other questions on RAM, restarts, zimbra version, etc. are?

[SamTzu]

As mentioned in the first post...
Nothing was done before the problem.
No restarts, updates, upgrades.
Ubuntu 6.06.1/Zimbra 5.04
>1Gb memory.

[SamTzu]

Here is when the problem started...
(Seems like the DB was somehow corrupted.)

Jul 28 00:42:01 server17 CRON[14712]: (pam_unix) session opened for user zimbra by (uid=0)
Jul 28 00:42:34 server17 zmmailboxdmgr[14884]: status requested
Jul 28 00:42:34 server17 zmmailboxdmgr[14884]: status OK
Jul 28 00:42:35 server17 clamd[10477]: reload db failed: Malformed database
Jul 28 00:42:35 server17 clamd[10477]: Terminating because of a fatal error.
Jul 28 00:42:35 server17 clamd[10477]: Pid file removed.
Jul 28 00:42:35 server17 clamd[10477]: --- Stopped at Mon Jul 28 00:42:35 2008
Jul 28 00:42:43 server17 zimbramon[14713]: 14713:info: 2008-07-28 00:42:05, STATUS: server17.nimet.info: antispam: Running
Jul 28 00:42:43 server17 zimbramon[14713]: 14713:info: 2008-07-28 00:42:05, STATUS: server17.nimet.info: antivirus: Running
Jul 28 00:42:43 server17 zimbramon[14713]: 14713:info: 2008-07-28 00:42:05, STATUS: server17.nimet.info: ldap: Running
Jul 28 00:42:44 server17 zimbramon[14713]: 14713:info: 2008-07-28 00:42:05, STATUS: server17.nimet.info: logger: Running
Jul 28 00:42:44 server17 zimbramon[14713]: 14713:info: 2008-07-28 00:42:05, STATUS: server17.nimet.info: mailbox: Running
Jul 28 00:42:44 server17 zimbramon[14713]: 14713:info: 2008-07-28 00:42:05, STATUS: server17.nimet.info: mta: Running
Jul 28 00:42:44 server17 zmmailboxdmgr[15033]: status requested
Jul 28 00:42:44 server17 zmmailboxdmgr[15033]: status OK
Jul 28 00:42:44 server17 zimbramon[14713]: 14713:info: 2008-07-28 00:42:05, STATUS: server17.nimet.info: snmp: Running
Jul 28 00:42:45 server17 zimbramon[14713]: 14713:info: 2008-07-28 00:42:05, STATUS: server17.nimet.info: spell: Running
Jul 28 00:42:45 server17 zimbramon[14713]: 14713:info: 2008-07-28 00:42:05, STATUS: server17.nimet.info: stats: Running
Jul 28 00:42:45 server17 CRON[14712]: (pam_unix) session closed for user zimbra
Jul 28 00:42:46 server17 zmmailboxdmgr[15077]: status requested
Jul 28 00:42:46 server17 zmmailboxdmgr[15077]: status OK
Jul 28 00:42:47 server17 zimbramon[28550]: 28550:err: Service status change: server17.nimet.info antivirus changed from running to stopped
Jul 28 00:42:47 server17 zimbramon[28550]: 28550:info: zmmtaconfig: Watchdog adding antivirus to restart list
Jul 28 00:42:48 server17 postfix/pickup[29129]: 7014AB70062: uid=1001 from=<zimbra>
Jul 28 00:42:48 server17 postfix/cleanup[14661]: 7014AB70062: message-id=<20080727214248.7014AB70062@server17.nimet.info >
Jul 28 00:42:48 server17 postfix/qmgr[10775]: 7014AB70062: from=<zimbra@server17.nimet.info>, size=494, nrcpt=2 (queue active)
Jul 28 00:42:48 server17 amavis[17474]: (17474-18) ESMTP::10024 /opt/zimbra/data/amavisd/tmp/amavis-20080727T130710-17474: <zimbra@server17.nimet.info> -> <admin@domainX.org>,<admin@server17.nimet.info> SIZE=494 Received: from server17.nimet.info ([127.0.0.1]) by localhost (server17.nimet.info [127.0.0.1]) (amavisd-new, port 10024) with ESMTP; Mon, 28 Jul 2008 00:42:48 +0300 (EEST)
Jul 28 00:42:48 server17 amavis[17474]: (17474-18) Checking: rfJ5Ocu1Dzv0 <zimbra@server17.nimet.info> -> <admin@domainX.org>,<admin@server17.nimet.info>
Jul 28 00:42:48 server17 amavis[17474]: (17474-18) ClamAV-clamd: Can't connect to INET socket 127.0.0.1:3310: Connection refused, retrying (1)
Jul 28 00:42:49 server17 amavis[17474]: (17474-18) (!)ClamAV-clamd: Can't connect to INET socket 127.0.0.1:3310: Connection refused, retrying (2)
Jul 28 00:42:55 server17 amavis[17474]: (17474-18) (!)run_av (ClamAV-clamd, built-in i/f): Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 75) line 310.
Jul 28 00:42:55 server17 amavis[17474]: (17474-18) (!!)ClamAV-clamd av-scanner FAILED: CODE(0x818e07c) Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 75) line 310. at (eval 75) line 511.
Jul 28 00:42:55 server17 amavis[17474]: (17474-18) (!!)WARN: all primary virus scanners failed, considering backups
Jul 28 00:42:55 server17 amavis[17474]: (17474-18) (!!)TROUBLE in check_mail: virus_scan FAILED: virus_scan: ALL VIRUS SCANNERS FAILED: ClamAV-clamd av-scanner FAILED: CODE(0x818e07c) Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 75) line 310. at (eval 75) line 511.
Jul 28 00:42:55 server17 amavis[17474]: (17474-18) (!)PRESERVING EVIDENCE in /opt/zimbra/data/amavisd/tmp/amavis-20080727T130710-17474
Jul 28 00:42:56 server17 postfix/smtp[14662]: 7014AB70062: to=<admin@domainX.org>, orig_to=<admin@server17.nimet.info>, relay=127.0.0.1[127.0.0.1]:10024, delay=7.6, delays=0.24/0/0.03/7.3, dsn=4.5.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.5.0 Error in processing, id=17474-18, virus_scan FAILED: virus_scan: ALL VIRUS SCANNERS FAILED: ClamAV-clamd av-scanner FAILED: CODE(0x818e07c) Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 75) line 310. at (eval 75) line 511. (in reply to end of DATA command))
Jul 28 00:42:56 server17 postfix/smtp[14662]: 7014AB70062: to=<admin@server17.nimet.info>, relay=127.0.0.1[127.0.0.1]:10024, delay=7.6, delays=0.24/0/0.03/7.3, dsn=4.5.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.5.0 Error in processing, id=17474-18, virus_scan FAILED: virus_scan: ALL VIRUS SCANNERS FAILED: ClamAV-clamd av-scanner FAILED: CODE(0x818e07c) Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 75) line 310. at (eval 75) line 511. (in reply to end of DATA command))
Jul 28 00:43:04 server17 postfix/smtpd[14171]: warning: 217.25.105.35: hostname a35-105.imatranet.fi verification failed: Name or service not known
Jul 28 00:43:04 server17 postfix/smtpd[14171]: connect from unknown[217.25.105.35]
Jul 28 00:43:04 server17 postfix/smtpd[14171]: disconnect from unknown[217.25.105.35]
Jul 28 00:43:14 server17 clamd[15272]: clamd daemon 0.92.1 (OS: linux-gnu, ARCH: i386, CPU: i686)
Jul 28 00:43:14 server17 clamd[15272]: Log file size limited to 20971520 bytes.
Jul 28 00:43:14 server17 clamd[15272]: Reading databases from /opt/zimbra/data/clamav/db
Jul 28 00:43:14 server17 clamd[15272]: Not loading PUA signatures.
Jul 28 00:43:22 server17 postfix/anvil[27393]: statistics: max connection rate 3/60s for (smtp:190.56.48.205) at Jul 28 00:35:39
Jul 28 00:43:22 server17 postfix/anvil[27393]: statistics: max connection count 3 for (smtp:190.56.48.205) at Jul 28 00:35:39
Jul 28 00:43:22 server17 postfix/anvil[27393]: statistics: max cache size 2 at Jul 28 00:33:42
Jul 28 00:43:33 server17 postfix/smtpd[14171]: connect from 98-bem-2.acn.waw.pl[62.121.81.98]
Jul 28 00:43:34 server17 clamd[15272]: Malformed database
Jul 28 00:43:34 server17 postfix/smtpd[14171]: NOQUEUE: reject: RCPT from 98-bem-2.acn.waw.pl[62.121.81.98]: 550 5.1.1 <kohdomainYgid@domainY.com>: Recipient address rejected: domainY.com; from=<modifies20@griffinserv.com> to=<kohdomainYgid@domainY.com> proto=ESMTP helo=<hp.home.aster.pl>
Jul 28 00:43:35 server17 postfix/smtpd[14171]: lost connection after DATA from 98-bem-2.acn.waw.pl[62.121.81.98]
Jul 28 00:43:35 server17 postfix/smtpd[14171]: disconnect from 98-bem-2.acn.waw.pl[62.121.81.98]
Jul 28 00:43:57 server17 postfix/smtpd[14171]: connect from 32-247-231-201.fibertel.com.ar[201.231.247.32]
Jul 28 00:43:58 server17 postfix/smtpd[14171]: 3DA4FB70064: client=32-247-231-201.fibertel.com.ar[201.231.247.32]
Jul 28 00:43:58 server17 postfix/cleanup[14661]: 3DA4FB70064: message-id=<1217194514.4389@levy.net>
Jul 28 00:43:58 server17 postfix/qmgr[10775]: 3DA4FB70064: from=<jtbeulahvv@levy.net>, size=1031, nrcpt=1 (queue active)
Jul 28 00:43:59 server17 amavis[22431]: (22431-08) ESMTP::10024 /opt/zimbra/data/amavisd/tmp/amavis-20080727T213249-22431: <jtbeulahvv@levy.net> -> <jyrki@domainY.com> SIZE=1031 BODY=8BITMIME Received: from server17.nimet.info ([127.0.0.1]) by localhost (server17.nimet.info [127.0.0.1]) (amavisd-new, port 10024) with ESMTP for <jyrki@domainY.com>; Mon, 28 Jul 2008 00:43:58 +0300 (EEST)
Jul 28 00:43:59 server17 amavis[22431]: (22431-08) Checking: EZAblNFteleX [201.231.247.32] <jtbeulahvv@levy.net> -> <jyrki@domainY.com>
Jul 28 00:43:59 server17 amavis[22431]: (22431-08) ClamAV-clamd: Can't connect to INET socket 127.0.0.1:3310: Connection refused, retrying (1)
Jul 28 00:44:00 server17 postfix/smtpd[14171]: disconnect from 32-247-231-201.fibertel.com.ar[201.231.247.32]
Jul 28 00:44:00 server17 amavis[22431]: (22431-08) (!)ClamAV-clamd: Can't connect to INET socket 127.0.0.1:3310: Connection refused, retrying (2)
Jul 28 00:44:01 server17 CRON[15296]: (pam_unix) session opened for user zimbra by (uid=0)
Jul 28 00:44:06 server17 amavis[22431]: (22431-08) (!)run_av (ClamAV-clamd, built-in i/f): Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 75) line 310.
Jul 28 00:44:06 server17 amavis[22431]: (22431-08) (!!)ClamAV-clamd av-scanner FAILED: CODE(0x818e07c) Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 75) line 310. at (eval 75) line 511.
Jul 28 00:44:06 server17 amavis[22431]: (22431-08) (!!)WARN: all primary virus scanners failed, considering backups
Jul 28 00:44:06 server17 amavis[22431]: (22431-08) (!!)TROUBLE in check_mail: virus_scan FAILED: virus_scan: ALL VIRUS SCANNERS FAILED: ClamAV-clamd av-scanner FAILED: CODE(0x818e07c) Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 75) line 310. at (eval 75) line 511.
Jul 28 00:44:06 server17 amavis[22431]: (22431-08) (!)PRESERVING EVIDENCE in /opt/zimbra/data/amavisd/tmp/amavis-20080727T213249-22431
Jul 28 00:44:06 server17 postfix/smtp[14662]: 3DA4FB70064: to=<jyrki@domainY.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=8.6, delays=0.67/0.01/0.08/7.9, dsn=4.5.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.5.0 Error in processing, id=22431-08, virus_scan FAILED: virus_scan: ALL VIRUS SCANNERS FAILED: ClamAV-clamd av-scanner FAILED: CODE(0x818e07c) Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 75) line 310. at (eval 75) line 511. (in reply to end of DATA command))
Jul 28 00:44:13 server17 zmmailboxdmgr[15413]: status requested
Jul 28 00:44:13 server17 zmmailboxdmgr[15413]: status OK
Jul 28 00:44:14 server17 zimbramon[28550]: 28550:info: zmmtaconfig: Failed to restart antivirus rc=256
Jul 28 00:44:17 server17 zimbramon[15297]: 15297:info: 2008-07-28 00:44:02, STATUS: server17.nimet.info: antispam: Running
Jul 28 00:44:17 server17 zimbramon[15297]: 15297:info: 2008-07-28 00:44:02, STATUS: server17.nimet.info: antivirus: Stopped
Jul 28 00:44:17 server17 zimbramon[15297]: 15297:info: 2008-07-28 00:44:02, STATUS: server17.nimet.info: amavisd: running
Jul 28 00:44:18 server17 zimbramon[15297]: 15297:info: 2008-07-28 00:44:02, STATUS: server17.nimet.info: ldap: Running
Jul 28 00:44:18 server17 zimbramon[15297]: 15297:info: 2008-07-28 00:44:02, STATUS: server17.nimet.info: logger: Running
Jul 28 00:44:18 server17 zimbramon[15297]: 15297:info: 2008-07-28 00:44:02, STATUS: server17.nimet.info: mailbox: Running
Jul 28 00:44:19 server17 zimbramon[15297]: 15297:info: 2008-07-28 00:44:02, STATUS: server17.nimet.info: mta: Running
Jul 28 00:44:19 server17 zimbramon[15297]: 15297:info: 2008-07-28 00:44:02, STATUS: server17.nimet.info: snmp: Running
Jul 28 00:44:19 server17 zimbramon[15297]: 15297:info: 2008-07-28 00:44:02, STATUS: server17.nimet.info: spell: Running
Jul 28 00:44:19 server17 zimbramon[15297]: 15297:info: 2008-07-28 00:44:02, STATUS: server17.nimet.info: stats: Running

[phoenix]

Try downloading the definitions files again, do the following:

su - zimbra
mkdir /tmp/clamdb
mv /opt/zimbra/clamav/db/* /tmp/clamdb
/opt/zimbra/clamav/bin/freshclam --config-file=/opt/zimbra/conf/freshclam.conf
zmamavisdctl stop
zmamavisdctl start

and see if it runs OK.

[SamTzu]

phoenix bill, you are the man!
Thank you very mucho grande.

zimbra@server17:/root$ zmcontrol status
Host server17.nimet.info
antispam Running
antivirus Running
ldap Running
logger Running
mailbox Running
mta Running
snmp Running
spell Running
stats Running


PS. It's zmamavisdctl with a d
/opt/zimbra/bin/zmamavisdctl start

[phoenix]

phoenix bill, you are the man!
Thank you very mucho grande.

zimbra@server17:/root$ zmcontrol status
Host server17.nimet.info
antispam Running
antivirus Running
ldap Running
logger Running
mailbox Running
mta Running
snmp Running
spell Running
stats Running

Gl;ad it's working, keep an eye on it and see if there's any further problems.

PS. It's zmamavisdctl with a d
/opt/zimbra/bin/zmamavisdctl start

I know that really, I'll have to keep an eye on my typing. Thanks for pointing it out, I've corrected it.

[davidkillingsworth]

Try downloading the definitions files again, do the following:

su - zimbra
mkdir /tmp/clamdb
mv /opt/zimbra/clamav/db/* /tmp/clamdb
/opt/zimbra/clamav/bin/freshclam --config-file=/opt/zimbra/conf/freshclam.conf
zmamavisdctl stop
zmamavisdctl start

and see if it runs OK.

This fixed my server this morning when clamav was causing the CPU utilization on the server to be very high and no mail was being delivered.

This is actually the 3rd time that the server has hung since upgrading to 7.2.0 a few weeks ago. It may not be related, but this never happened before upgrading.

Usually, if we just restart the server, everything is ok, but this time even a server reboot would not get the server back.

Running the commands above got the server working again.