status not updating properly

[useopenid]

I've got a problem where the service status is not getting updated properly; logswatch and swatch are running:


0 31582 18076 1 16 0 6720 4804 wait S ? 0:00 /usr/bin/perl /opt/zimbra/libexec/logswatch --config-file=/opt/zimbra/conf/logswatchrc --use-cpan-file-tail --pid-file=/opt/zimbra/
0 31582 18085 18076 18 0 8744 6768 pipe_w S ? 2:27 /usr/bin/perl /tmp/.swatch_script.18076
0 31582 18093 18085 15 0 8928 4768 - S ? 7:33 /usr/bin/perl /opt/zimbra/libexec/zmlogger
0 31582 18351 1 19 0 6728 4820 wait S ? 0:00 /usr/bin/perl /opt/zimbra/libexec/swatch --config-file=/opt/zimbra/conf/swatchrc --use-cpan-file-tail --script-dir=/tmp -t /var/log
0 31582 18372 18351 15 0 8884 6776 - S ? 1:20 /usr/bin/perl /tmp/.swatch_script.18351


In looking at the code for zmlogger, I see that it's looking at the STATUS lines in /var/log/zimbra.log, and those are updating properly:


Apr 17 18:06:19 zmail01-store01 zimbramon[16128]: 16128:info: 2009-04-17 18:06:01, STATUS: zmail01-store01.server.peak.org: imapproxy: Running
Apr 17 18:06:19 zmail01-store01 zimbramon[16128]: 16128:info: 2009-04-17 18:06:01, STATUS: zmail01-store01.server.peak.org: logger: Running
Apr 17 18:06:19 zmail01-store01 zimbramon[16128]: 16128:info: 2009-04-17 18:06:01, STATUS: zmail01-store01.server.peak.org: mailbox: Running
Apr 17 18:06:19 zmail01-store01 zimbramon[16128]: 16128:info: 2009-04-17 18:06:01, STATUS: zmail01-store01.server.peak.org: snmp: Running
Apr 17 18:06:19 zmail01-store01 zimbramon[16128]: 16128:info: 2009-04-17 18:06:01, STATUS: zmail01-store01.server.peak.org: spell: Running
Apr 17 18:06:19 zmail01-store01 zimbramon[16128]: 16128:info: 2009-04-17 18:06:01, STATUS: zmail01-store01.server.peak.org: stats: Running


but the database is not being updated:


mysql> select * from service_status;
+---------------------------------+-----------+---------------------+--------+-----------------+
| server | service | time | status | loghostname |
+---------------------------------+-----------+---------------------+--------+-----------------+
| zmail01-store01.server.peak.org | stats | 2009-04-17 10:20:03 | 1 | zmail01-store01 |
| zmail01-store01.server.peak.org | spell | 2009-04-17 10:20:03 | 1 | zmail01-store01 |
| zmail01-store01.server.peak.org | snmp | 2009-04-17 10:20:03 | 1 | zmail01-store01 |
| zmail01-store01.server.peak.org | mailbox | 2009-04-17 10:20:03 | 1 | zmail01-store01 |
| zmail01-store01.server.peak.org | logger | 2009-04-17 10:20:03 | 1 | zmail01-store01 |
| zmail01-store01.server.peak.org | imapproxy | 2009-04-17 10:20:03 | 1 | zmail01-store01 |


I'm turning on debug in zmlogger, but thought I would post here now in case something like this has been seen before...

[useopenid]

When I restarted zimbra to get the zmlogger debug to take effect, the service status updated as I expected it would, however the web page still says "data is stale" after all the entries, even though the time stamps are within a couple minutes of current???

[uxbod]

su - zimbra
logmysql
use zimbra_logger;
select * from mta order by arrive_time desc limit 10;

Are you seeing upto date data ??

[useopenid]

It does appear to be getting updated, but running way behind at the moment...from 25 minutes late an hour ago, to 40 minutes behind now...

[phoenix]

Originally Posted by useopenid

It does appear to be getting updated, but running way behind at the moment...from 25 minutes late an hour ago, to 40 minutes behind now...

Are you running Zimbra in a VM?

[useopenid]

Yes, but it's the sole guest on a dual 3ghz processor blade. It's a vm for ease of management.

[phoenix]

Originally Posted by useopenid

Yes, but it's the sole guest on a dual 3ghz processor blade. It's a vm for ease of management.

That's not the point of my question. Is this VMware by any chance? If it is the timer of the guest can drift and cause you problems. Have you followed the Best Practice guide from the VMware KB?

BTW, could you please update your forum profile with the output of the following command:

zmcontrol -v

[phoenix]

One additional question. This line appears to show the imapproxy running:

Originally Posted by useopenid

Apr 17 18:06:19 zmail01-store01 zimbramon[16128]: 16128:info: 2009-04-17 18:06:01, STATUS: zmail01-store01.server.peak.org: imapproxy: Running

Is this a single server or a multi-server install?

[useopenid]

Multiserver, each one is the sole guest on esxi. We have two ldap servers, in replication, two stores and two mtas, each of the latter running proxy. We intend to scale this up to around 20-30,000 mailboxes with a number of inexpensive systems rather than an expensive piece of bigiron. It should still be cheaper, and in any case, should have better failure tolerance.

At the moment, we're running ntpdate from cron every minute to keep the time in sync; that link looks like it might have some good options --- I don't remember seeing it when I last looked into the issue, though it's less of a problem for us on esx than it is on vmware server.

[hodd]

Originally Posted by uxbod

su - zimbra
logmysql
use zimbra_logger;
select * from mta order by arrive_time desc limit 10;

Are you seeing upto date data ??

We're having the exact same problem with our server, and running this showed that the stats haven't been updated since Dec. 6th, 2008. Is there a way to flush this out and get the stats updated again?

Thanks!