LinkBack URL
About LinkBacks
server stats behaviour inconsistent across multiple servers
Hi all,
I've been searching google and this forum for someone with a system showing similar symptoms and who might have come to a solution:
The problem: We have got multiple (2) mailbox servers all running 6.0.4_GA_2038.UBUNTU8_64 UBUNTU8_64 FOSS. The names are "mail" and "mail-eu".
The main server (mail) handles all incoming mail and sends it to the appropriate store.
Everything work fine from an operational perspective. The monitoring causes me some grief, since the system-wide server statistics display okay and seem to be correct and so are the stats for "mail-eu". But when I access the server statistics for "mail" the panels "Message count", "Message volume", "AS/AV activity" and "disk" contain nothing but the headers.
Under the system wide stats "Advanced" tab, I can select the server "mail" and the group "zmmmtastats". This will plot some graphs but these don't seem to be correct, at least not all of them. So is mta_volume going up an down but mta_count shows a flat line at "0". That doesn't seem right.
I read through all posts of After update to 6.0 GA Statistic does not show anything but all actions suggested didn't fix the issue.
According to the wiki Logger (ZCS 6.0.x and later) - Zimbra :: Wiki , here's my output:
zmloggerctl status; echo $?; ps ax | grep zmlogger
/opt/zimbra/libexec/zmrrdfetch -f zmmtastatsCode:0 15264 ? D 1:07 /usr/bin/perl /opt/zimbra/libexec/zmlogger 15335 ? S 0:00 zmlogger: zmrrdfetch: server 31113 pts/1 S+ 0:00 grep zmlogger
/opt/zimbra/libexec/zmrrdfetch -c -f zmmtastats [/CODE]Code:Host: mail.magicmemories.co.nz timestamp,mta_count,clam_events,mta_delay,mta_volume,filter_count,filter_misc,filter_virus,filter_spam,sendmail_events 1270358190,0.0363742387648574,0.00166840634667415,,113.3510488042,0.0181871193824287,0,0,0,0 1270358220,0.0363742387648574,0.00166840634667415,,113.3510488042,0.018187119382 4287,0,0,0,0 1270358250,0.0363742387648574,0.00166840634667415,,113.3510488042,0.018187119382 4287,0,0,0,0 1270358280,0.0363742387648574,0.00166840634667415,,113.3510488042,0.018187119382 4287,0,0,0,0 1270358310,0.0363742387648574,0.00166840634667415,,113.3510488042,0.018187119382 4287,0,0,0,0 .. and so forth.
grep -w MTA /var/log/zimbra-stats.logCode:Host: mail.magicmemories.co.nz timestamp,mta_count,clam_events,mta_delay,mta_volume,filter_count,filter_misc,filter_virus,filter_spam,sendmail_events 1270358400,0.0363742387648574,0.00166840634667415,,113.3510488042,0.0181871193824287,0,0,0,0 1270358700,0.0136038159971582,1.8929123865887e-05,,868.075042938541,0.0068019079985791,0,0,0,0 1270359000,0.0136038159971582,1.8929123865887e-05,,868.075042938541,0.0068019079985791,0,0,0,0 1270359300,,,,,,,,, 1270359600,,,,,,,,, ... and so forth.
tail /var/log/zimbra-stats.logCode:Apr 5 06:50:09 wanaka zimbramon[9298]: 9298:info: MTA: mail-eu.magicmemories.com: filter_misc,clam_events,mta_delay,mta_volume,filter_virus,filter_count,mta_count,filter_spam,sendmail_events:: 0,0,0,0,0,0,0,0,0 Apr 5 06:50:09 wanaka zimbramon[9298]: 9298:info: MTA: mail.magicmemories.co.nz: filter_misc,clam_events,mta_delay,mta_volume,filter_virus,filter_count,mta_count,filter_spam,sendmail_events:: 0,1,0.5,9748,0,2,4,0,0 Apr 5 07:00:06 wanaka zimbramon[21538]: 21538:info: MTA: mail-eu.magicmemories.com: filter_misc,clam_events,mta_delay,mta_volume,filter_virus,filter_count,mta_count,filter_spam,sendmail_events:: 0,0,0,0,0,0,0,0,0 Apr 5 07:00:06 wanaka zimbramon[21538]: 21538:info: MTA: mail.magicmemories.co.nz: filter_misc,clam_events,mta_delay,mta_volume,filter_virus,filter_count,mta_count,filter_spam,sendmail_events:: 0,0,0.75,10709,0,2,4,0,0 Apr 5 07:10:06 wanaka zimbramon[2868]: 2868:info: MTA: mail-eu.magicmemories.com: filter_misc,clam_events,mta_delay,mta_volume,filter_virus,filter_count,mta_count,filter_spam,sendmail_events:: 0,0,0,0,0,0,0,0,0 .. and so forth
zmsoap -z GetLoggerStatsRequest stats/@name=zmmtastats | head -20Code:Apr 5 17:21:03 wanaka zimbramon[4810]: :::05E2B05A-4073-11DF-9BE8-B71D20A5F8DB:::0,0,,5653016,0,0,0,0,4,5,9,2,610064 Apr 5 17:21:03 wanaka zimbramon[4794]: 4794:info: zmstat proc.csv: timestamp, system, user, sys, idle, iowait, mailbox, mailbox-total-cpu, mailbox-utime, mailbox-stime, mailbox-totalMB, mailbox-rssMB, mailbox-sharedMB, mailbox-process-count, mysql, mysql-total-cpu, mysql-utime, mysql-stime, mysql-totalMB, mysql-rssMB, mysql-sharedMB, mysql-process-count, convertd, convertd-total-cpu, convertd-utime, convertd-stime, convertd-totalMB, convertd-rssMB, convertd-sharedMB, convertd-process-count, ldap, ldap-total-cpu, ldap-utime, ldap-stime, ldap-totalMB, ldap-rssMB, ldap-sharedMB, ldap-process-count, postfix, postfix-total-cpu, postfix-utime, postfix-stime, postfix-totalMB, postfix-rssMB, postfix-sharedMB, postfix-process-count, amavis, amavis-total-cpu, amavis-utime, amavis-stime, amavis-totalMB, amavis-rssMB, amavis-sharedMB, amavis-process-:::05F3C5DE-4073-11DF-88AD-A71D20A5F8DB::: Apr 5 17:21:04 wanaka zimbramon[4794]: :::05F3C5DE-4073-11DF-88AD-A71D20A5F8DB:::count, clam, clam-total-cpu, clam-utime, clam-stime, clam-totalMB, clam-rssMB, clam-sharedMB, clam-process-count:: 04/05/2010 17:21:00, system, 17.6, 1.4, 78.9, 2.0, mailbox, 0.2, 0.2, 0.0, 3388.7, 3065.9, 15.2, 1, mysql, 0.0, 0.0, 0.0, 2257.1, 892.7, 5.0, 1, convertd, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0, ldap, 0.0, 0.0, 0.0, 384.7, 102.6, 9.0, 1, postfix, 0.0, 0.0, 0.0, 81.4, 4.5, 3.4, 34, amavis, 0.2, 0.2, 0.0, 206.1, 91.5, 3.6, 11, clam, 0.5, 0.5, 0.0, 334.2, 148.8, 1.4, 2 Apr 5 17:21:04 wanaka mailboxd: INFO [soap.csv] [] slogger - soap.csv: timestamp,command,exec_count,exec_ms_avg:: 04/05/2010 16:50:36,AuthRequest,1,92 Apr 5 17:21:05 wanaka mailboxd: INFO [imap.csv] [] slogger - imap.csv: timestamp,command,exec_count,exec_ms_avg:: 04/05/2010 16:00:59,CAPABILITY,2,0 Apr 5 17:21:05 wanaka zimbramon[4804]: :::CA601FCA-4060-11DF-A653-B11D20A5F8DB:::93, 0.67, 1947.20, 20, 58416, 0.00, 0.00, 0.00, 0, 0, 0.00, 0.00, 0.00, 0, 0 Apr 5 17:21:06 wanaka mailboxd: INFO [imap.csv] [] slogger - imap.csv: timestamp,command,exec_count,exec_ms_avg:: 04/05/2010 16:18:00,CLOSE,1,0 Apr 5 17:21:06 wanaka mailboxd: INFO [threads.csv] [] slogger - threads.csv: timestamp,btpool,pool,LmtpServer,ImapServer,ImapSSLServer,Pop3Server,Pop3SSLServer,ScheduledTask,Timer,AnonymousIoService,CloudRoutingReaderThread,GC,SocketAcceptor,Thread,other,total:: 04/05/2010 17:11:57,11,2,3,13,1,1,1,0,9,16,0,0,4,4,25,90 Apr 5 17:21:07 wanaka zimbramon[4814]: 4814:info: zmstat mtaqueue.csv: timestamp, KBytes, requests:: 04/05/2010 15:58:13, 0, 0 Apr 5 17:21:07 wanaka mailboxd: INFO [threads.csv] [] slogger - threads.csv: timestamp,btpool,pool,LmtpServer,ImapServer,ImapSSLServer,Pop3Server,Pop3SSLServer,ScheduledTask,Timer,AnonymousIoService,CloudRoutingReaderThread,GC,SocketAcceptor,Thread,other,total:: 04/05/2010 15:39:53,10,2,1,13,1,1,1,0,9,16,0,0,4,4,25,87
zmsoap -z GetLoggerStatsRequest stats/@name=zmmtastats @limit=1 | head -50PHP Code:<GetLoggerStatsResponse xmlns="urn:zimbraAdmin">
<hostname hn="caspian.magicmemories.co.nz">
<stats name="zmmtastats"/>
</hostname>
<hostname hn="mail-eu.magicmemories.com">
<stats name="zmmtastats">
<values t="1270358520">
<stat name="filter_misc" value="0"/>
<stat name="clam_events" value="0"/>
<stat name="mta_delay" value=""/>
<stat name="mta_volume" value="0"/>
<stat name="filter_virus" value="0"/>
<stat name="filter_count" value="0"/>
<stat name="mta_count" value="0"/>
<stat name="filter_spam" value="0"/>
<stat name="sendmail_events" value="0"/>
</values>
<values t="1270358550">
<stat name="filter_misc" value="0"/>
<stat name="clam_events" value="0"/>
Further server names:PHP Code:<GetLoggerStatsResponse xmlns="urn:zimbraAdmin">
<hostname hn="caspian.magicmemories.co.nz">
<stats name="zmmtastats"/>
</hostname>
<hostname hn="mail-eu.magicmemories.com">
<stats name="zmmtastats">
<values t="1270358700">
<stat name="filter_misc" value="0"/>
<stat name="clam_events" value="0"/>
<stat name="mta_delay" value=""/>
<stat name="mta_volume" value="0"/>
<stat name="filter_virus" value="0"/>
<stat name="filter_count" value="0"/>
<stat name="mta_count" value="0"/>
<stat name="filter_spam" value="0"/>
<stat name="sendmail_events" value="0"/>
</values>
<values t="1270359000">
<stat name="filter_misc" value="0"/>
<stat name="clam_events" value="0"/>
<stat name="mta_delay" value=""/>
<stat name="mta_volume" value="0"/>
<stat name="filter_virus" value="0"/>
<stat name="filter_count" value="0"/>
<stat name="mta_count" value="0"/>
<stat name="filter_spam" value="0"/>
<stat name="sendmail_events" value="0"/>
</values>
<values t="1270359900">
<stat name="filter_misc" value="0"/>
<stat name="clam_events" value="0"/>
<stat name="mta_delay" value=""/>
<stat name="mta_volume" value="0"/>
<stat name="filter_virus" value="0"/>
<stat name="filter_count" value="0"/>
<stat name="mta_count" value="0"/>
<stat name="filter_spam" value="0"/>
<stat name="sendmail_events" value="0"/>
</values>
<values t="1270360200">
<stat name="filter_misc" value="0"/>
<stat name="clam_events" value="0"/>
<stat name="mta_delay" value=""/>
<stat name="mta_volume" value="0"/>
<stat name="filter_virus" value="0"/>
<stat name="filter_count" value="0"/>
<stat name="mta_count" value="0"/>
<stat name="filter_spam" value="0"/>
<stat name="sendmail_events" value="0"/>
</values>
wanaka .. that's the machine name hosting "mail".
caspian ... the original server hosting ZCS 5.x, which was upgraded to 6.0.4 and then moved to wanaka. So did the DNS entry for mail move from pointing to caspian to point to wanaka.
No need to say, that I'd like to get the stats working to identify bottlenecks before they occur and for general system monitoring.
Any help or ideas, why the stats are not showing up?
/Stefan
