Results 1 to 6 of 6

Thread: server stats are not working

  1. #1
    stophi is offline Active Member
    Join Date
    Feb 2011
    Posts
    27
    Rep Power
    4

    Default server stats are not working

    hi.

    after upgrading to network-edition server stats in admin-interface are not working any more.

    Code:
    zmcontrol -v
    Release 8.0.1_GA_5438.SLES11_64_20121105164455 SLES11_64 NETWORK edition.
    i searched the forum but could not solve the problem. here are some information:

    Code:
    cat /etc/SuSE-release
    SUSE Linux Enterprise Server 11 (x86_64)
    VERSION = 11
    PATCHLEVEL = 2
    Code:
    cat /etc/profile.local
    ulimit -v unlimited
    ulimit -m unlimited

    Code:
    zmcontrol status
    Host zimbraserver.com
            antispam                Running
            antivirus               Running
            convertd                Running
            ldap                    Running
            logger                  Running
            mailbox                 Running
            memcached               Running
            mta                     Running
            proxy                   Running
            snmp                    Running
            spell                   Running
            stats                   Running
            zmconfigd               Running

    Code:
    cat /etc/syslog-ng/syslog-ng.conf | grep zimbra
                                   filter(f_iptables) and not filter(zimbra_local0) and not filter(zimbra_local1)
                                   and not filter(zimbra_auth) and not filter(zimbra_mail); };
                                   not filter(zimbra_local0) and not filter(zimbra_local1) and not
                                   filter(zimbra_auth) and not filter(zimbra_mail); };
    filter zimbra_local0 { facility(local0); }; # zimbra
    filter zimbra_local1 { facility(local1); }; # zimbra
    filter zimbra_auth { facility(auth); }; # zimbra
    filter zimbra_mail { facility(mail); }; # zimbra
    destination zimbra_mail { udp("1.1.1.1" port(514) ); }; # zimbra
    destination zimbra_local1 { udp("1.1.1.1" port(514) ); }; # zimbra
    destination zimbra_local0 { udp("1.1.1.1" port(514) ); }; # zimbra
    destination zimbra_auth { udp("1.1.1.1" port(514) ); }; # zimbra
    log { source(src); filter(zimbra_mail); destination(zimbra_mail); }; # zimbra
    log { source(src); filter(zimbra_local0); destination(zimbra_local0); }; # zimbra
    log { source(src); filter(zimbra_local1); destination(zimbra_local1); }; # zimbra
    log { source(src); filter(zimbra_auth); destination(zimbra_auth); }; # zimbra



    some hints what might be wrong


    Code:
    /opt/zimbra/libexec/zmrrdfetch -f zmmtastats
    DBD::SQLite::db prepare failed: no such table: rrds at /opt/zimbra/libexec/zmrrdfetch line 57.
    Can't call method "bind_param" on an undefined value at /opt/zimbra/libexec/zmrrdfetch line 62.

    Code:
    /var/log/zimbra.log is empty
    Code:
    tail /var/log/messages
    Dec  6 16:11:28 vvx-mail-01 syslog-ng[58510]: I/O error occurred while writing; fd='4', error='Connection refused (111)'
    Dec  6 16:11:28 vvx-mail-01 syslog-ng[58510]: Connection broken; time_reopen='60'
    Dec  6 16:11:45 vvx-mail-01 syslog-ng[58510]: I/O error occurred while writing; fd='4', error='Connection refused (111)'
    Dec  6 16:11:45 vvx-mail-01 syslog-ng[58510]: Connection broken; time_reopen='60'
    Dec  6 16:11:49 vvx-mail-01 syslog-ng[35193]: I/O error occurred while writing; fd='5', error='Connection refused (111)'
    Dec  6 16:11:49 vvx-mail-01 syslog-ng[35193]: Connection broken; time_reopen='60'
    Dec  6 16:11:49 vvx-mail-01 syslog-ng[35193]: I/O error occurred while writing; fd='5', error='Connection refused (111)'
    Dec  6 16:11:49 vvx-mail-01 syslog-ng[35193]: Connection broken; time_reopen='60'
    Dec  6 16:11:53 vvx-mail-01 syslog-ng[58510]: I/O error occurred while writing; fd='4', error='Connection refused (111)'
    Dec  6 16:11:53 vvx-mail-01 syslog-ng[58510]: Connection broken; time_reopen='60'
    Last edited by stophi; 12-06-2012 at 08:19 AM.

  2. #2
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,480
    Rep Power
    56

    Default

    You shouldn't have memcached or the proxy running on a single server, search the forums for details on how to stop & disable those services and reset the ports. You could also reinitialize the logger, you might also have a problem with syslog-ng or you could try some of the solutions mentioned in these forum posts: site:zimbra.com +"/opt/zimbra/libexec/zmrrdfetch line" - Yahoo! Search Results
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  3. #3
    stophi is offline Active Member
    Join Date
    Feb 2011
    Posts
    27
    Rep Power
    4

    Default

    bill, thank you for your feedback.

    but I could not solve it yet.


    1) You shouldn't have memcached or the proxy running on a single server

    i disable memcached - but when I disable proxy imap is not working anymore. so i enable proxy again.

    Code:
    zmmemcachedctl stop
    zmprov ms `zmhostname` -zimbraServiceEnabled memcached
    zmproxyctl stop
    zmprov ms `zmhostname` -zimbraServiceEnabled proxy
    
    -> imap not working anymore
    zmproxyctl start
    zmprov ms `zmhostname` +zimbraServiceEnabled proxy


    2) You could also reinitialize the logger

    Code:
    /opt/zimbra/libexec/zmsyslogsetup
    updateSyslogNG: Updating /etc/syslog-ng/syslog-ng.conf...done.

    3) you might also have a problem with syslog-ng
    Code:
    tail -n 20 /etc/syslog-ng/syslog-ng.conf
    
    #
    # Warnings (except iptables) in one file:
    #
    destination warn { file("/var/log/warn" fsync(yes)); };
    log { source(src); filter(f_warn); destination(warn); };
    
    
    filter zimbra_local0 { facility(local0); }; # zimbra
    filter zimbra_local1 { facility(local1); }; # zimbra
    filter zimbra_auth { facility(auth); }; # zimbra
    filter zimbra_mail { facility(mail); }; # zimbra
    destination zimbra_mail { udp("91.195.95.131" port(514) ); }; # zimbra
    destination zimbra_local1 { udp("91.195.95.131" port(514) ); }; # zimbra
    destination zimbra_local0 { udp("91.195.95.131" port(514) ); }; # zimbra
    destination zimbra_auth { udp("91.195.95.131" port(514) ); }; # zimbra
    log { source(src); filter(zimbra_mail); destination(zimbra_mail); }; # zimbra
    log { source(src); filter(zimbra_local0); destination(zimbra_local0); }; # zimbra
    log { source(src); filter(zimbra_local1); destination(zimbra_local1); }; # zimbra
    log { source(src); filter(zimbra_auth); destination(zimbra_auth); }; # zimbra


    4) could try some of the solutions mentioned in these forum posts:

    I found this posts which seems to be the same problem:
    [SOLVED] Logger problems after upgrade 5.0.9 to 6.0.6

    but it already failed at step 2:
    Code:
    zimbra@vvx-mail-01:/root> /opt/zimbra/libexec/zmlogger&
    [1] 11261
    zimbra@vvx-mail-01:/root> zmrrdfetch server pid: 11295
    DBD::SQLite::db selectrow_array failed: no such table: config at /opt/zimbra/libexec/zmlogger line 409.
    Schema version not found, upgrading to version 4 at /opt/zimbra/libexec/zmlogger line 412.
    
    
    [1]+  Stopped                 /opt/zimbra/libexec/zmlogger



    5) troubleshooting logs
    Logger (ZCS 6.0.x and later) - Zimbra :: Wiki

    Code:
    zmloggerctl status; echo $?; ps ax | grep zmlogger
    0
    20605 pts/1    S+     0:00 grep zmlogger
    
    /opt/zimbra/libexec/zmrrdfetch -f zmmtastats
    no structure found for zmmtastats at /opt/zimbra/libexec/zmrrdfetch line 239.
    
    /opt/zimbra/libexec/zmrrdfetch -c -f zmmtastats
    no structure found for zmmtastats at /opt/zimbra/libexec/zmrrdfetch line 239.
    
    grep -w MTA /var/log/zimbra-stats.log
    0
    
    tail /var/log/zimbra-stats.log
    0
    
    zmsoap -z GetLoggerStatsRequest stats/@name=zmmtastats | head -20
    ERROR: service.FAILURE (system failure: Unable to read logger stats)
    
    zmsoap -z GetLoggerStatsRequest stats/@name=zmmtastats @limit=1 | head -50
    ERROR: service.FAILURE (system failure: Unable to read logger stats)

    so I checked this posts without success:
    [SOLVED] ZIMBRA LOG error
    [SOLVED] Zimbra admin, no services, no stats running.
    [SOLVED] Unable to read logger status
    [SOLVED] After install server status shows errors for all services

  4. #4
    stophi is offline Active Member
    Join Date
    Feb 2011
    Posts
    27
    Rep Power
    4

    Default

    ok - now logging to /var/log/zimbra.log and /var/log/zimbra-stats.log works again - but not the rest.

    what have I done?

    when I run following command:
    Code:
    /opt/zimbra/libexec/zmsyslogsetup
    then /etc/syslog-ng/syslog-ng.conf looks like this:
    Code:
    cat syslog-ng.conf | grep zimbra
                                   filter(f_iptables) and not filter(zimbra_local0) and not filter(zimbra_local1)
                                   and not filter(zimbra_auth) and not filter(zimbra_mail); };
                                   not filter(zimbra_local0) and not filter(zimbra_local1) and not
                                   filter(zimbra_auth) and not filter(zimbra_mail); };
    filter zimbra_local0 { facility(local0); }; # zimbra
    filter zimbra_local1 { facility(local1); }; # zimbra
    filter zimbra_auth { facility(auth); }; # zimbra
    filter zimbra_mail { facility(mail); }; # zimbra
    destination zimbra_mail { udp("X.X.X.X" port(514) ); }; # zimbra
    destination zimbra_local1 { udp("X.X.X.X" port(514) ); }; # zimbra
    destination zimbra_local0 { udp("X.X.X.X" port(514) ); }; # zimbra
    destination zimbra_auth { udp("X.X.X.X" port(514) ); }; # zimbra
    log { source(src); filter(zimbra_mail); destination(zimbra_mail); }; # zimbra
    log { source(src); filter(zimbra_local0); destination(zimbra_local0); }; # zimbra
    log { source(src); filter(zimbra_local1); destination(zimbra_local1); }; # zimbra
    log { source(src); filter(zimbra_auth); destination(zimbra_auth); }; # zimbra
    restarting syslog tells me this:
    Code:
    rcsyslog restart
    Shutting down syslog services                                                                                                                                                                    done
    Starting syslog servicesDuplicate stats counter; counter='udp(AF_INET(X.X.X.X:514))'
    Duplicate stats counter; counter='udp(AF_INET(X.X.X.X:514))'
    Duplicate stats counter; counter='udp(AF_INET(X.X.X.X:514))'


    I changed syslog-ng.conf to this:
    Code:
    cat syslog-ng.conf | grep zimbra
                                   filter(f_iptables) and not filter(zimbra_local0) and not filter(zimbra_local1)
                                   and not filter(zimbra_auth) and not filter(zimbra_mail); };
                                   not filter(zimbra_local0) and not filter(zimbra_local1) and not
                                   filter(zimbra_auth) and not filter(zimbra_mail); };
    filter zimbra_local0 { facility(local0); }; # zimbra
    filter zimbra_local1 { facility(local1); }; # zimbra
    filter zimbra_auth { facility(auth); }; # zimbra
    filter zimbra_mail { facility(mail); }; # zimbra
    destination zimbra_mail { file("/var/log/zimbra.log" owner("zimbra")); }; # zimbra
    destination zimbra_local1 { file("/var/log/zimbra-stats.log" owner("zimbra")); }; # zimbra
    destination zimbra_local0 { file("/var/log/zimbra.log" owner("zimbra")); }; # zimbra
    destination zimbra_auth { file("/var/log/zimbra.log" owner("zimbra")); }; # zimbra
    log { source(src); filter(zimbra_mail); destination(zimbra_mail); }; # zimbra
    log { source(src); filter(zimbra_local0); destination(zimbra_local0); }; # zimbra
    log { source(src); filter(zimbra_local1); destination(zimbra_local1); }; # zimbra
    log { source(src); filter(zimbra_auth); destination(zimbra_auth); }; # zimbra

    restarting syslog also is better now:
    Code:
    rcsyslog restart
    Shutting down syslog services                                                                                                                                                                    done
    Starting syslog services         done


    now troubleshooting-logs give this:

    Code:
    zmloggerctl status; echo $?; ps ax | grep zmlogger
    0
    45492 ?        R      0:00 /usr/bin/perl /opt/zimbra/libexec/zmlogger
    45527 ?        S      0:00 zmlogger: zmrrdfetch: server
    45582 pts/1    S+     0:00 grep zmlogger
    
    /opt/zimbra/libexec/zmrrdfetch -f zmmtastats
    no structure found for zmmtastats at /opt/zimbra/libexec/zmrrdfetch line 239.
    
    /opt/zimbra/libexec/zmrrdfetch -c -f zmmtastats
    no structure found for zmmtastats at /opt/zimbra/libexec/zmrrdfetch line 239.
    
    grep -w MTA /var/log/zimbra-stats.log
    -> nothing
    
    tail /var/log/zimbra-stats.log
    Dec 10 12:13:30 vvx-mail-01 zimbramon[6959]: 6959:info: :::A102CD9E-42BA-11E2-A3DC-F4F86A583D9E:::0,0,0,0,0,0,0,0,0,0,0,0,79562,0,0,0,0,0,0,0,0,0,0,0,0,8988,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,21232,4807,0,0,0,0,0,0,0,0,0,0,0,0,0,12935,0,0,0,0,0,80369,0,0,72876,241098,0,0,0,0,0,7,0,0,0,0,0,0,0,0,0,0,130,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,593,0,0,0,0,7,0,0,0,0,0,0,0,0,0,0,0,0,0,43408,3,0,0,0,0,0,0,OFF,602,3,5,546,0,0,0,1,243279,4627,0,0,63,394660,0,3794951,23935,132854,180774,661,0,0,61208,203670,38739,0,158550,0,876,39615,0,3885,41,15322115,40398,0,785130,16198,0,0,0,733302784,41495,64244,2648286208,79275,1564,ON,0,113809,4220,4962,0,0,50223104,16384,1277,44623,79275,0,0,0,0,1,7479,33205,4148566,61208,0,0,6698,1,0,0,2,0,0.000000,8,0,17,0,475,525,537,482,532,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,566540,566540,AUTH_MASTER,0,0,9712,0,585,0.000,0,0,0,OFF,0,52,0,1318,12967,1,0,0,:::A102D424-42BA-11E2-A3DC-F4F86A583D9E:::
    Dec 10 12:13:30 vvx-mail-01 zimbramon[6959]: :::A102D424-42BA-11E2-A3DC-F4F86A583D9E:::0,,,0,0,0,0,0,0,0,0,0,NONE,0,0,0,0,0,0,0,,175824,0,0,0,0,0,8,8,1,7869,7869
    Dec 10 12:13:31 vvx-mail-01 zimbramon[60376]: 60376:info: zmstat fd.csv: timestamp, fd_count, mailboxd_fd_count:: 12/10/2012 12:13:31, 5408, 1725
    Dec 10 12:13:34 vvx-mail-01 zimbramon[59914]: 59914:info: zmstat convertd.csv: timestamp, utime, stime, cputime, rchar, wchar, read_bytes, write_bytes, rss, processes, threads:: 12/10/2012 12:13:34, 0, 0, 0, 0, 0, 0, 0, 62284, 6, 6
    Dec 10 12:13:35 vvx-mail-01 zimbramon[6953]: 6953:info: zmstat io.csv: timestamp, xvda:tps, xvda:kB_read/s, xvda:kB_wrtn/s, xvda:kB_read, xvda:kB_wrtn:: 12/10/2012 12:13:35, 4.57, 0.13, 45.87, 4, 1376
    Dec 10 12:13:35 vvx-mail-01 zimbramon[6949]: 6949:info: zmstat io-x.csv: timestamp, xvda:rrqm/s, xvda:wrqm/s, xvda:r/s, xvda:w/s, xvda:rkB/s, xvda:wkB/s, xvda:avgrq-sz, xvda:avgqu-sz, xvda:await, xvda:svctm, xvda:%util:: 12/10/2012 12:13:35, 0.00, 3.70, 0.03, 4.30, 0.13, 43.60, 20.18, 0.04, 8.49, 1.51, 0.65
    Dec 10 12:13:36 vvx-mail-01 zimbramon[59916]: 59916:info: zmstat mtaqueue.csv: timestamp, KBytes, requests:: 12/10/2012 12:13:36, 146, 3
    Dec 10 12:13:42 vvx-mail-01 zimbramon[6965]: 6965:info: zmstat nginx.csv: timestamp, utime, stime, cputime, rchar, wchar, read_bytes, write_bytes, rss, processes, threads:: 12/10/2012 12:13:42, 1, 3, 4, 52503, 86904, 0, 4096, 34092, 5, 5
    Dec 10 12:13:44 vvx-mail-01 zimbramon[6947]: 6947:info: zmstat vm.csv: timestamp, r, b, swpd, free, buff, cache, si, so, bi, bo, in, cs, us, sy, id, wa, st, MemTotal, MemFree, Buffers, Cached, SwapCached, Active, Inactive, Active(anon), Inactive(anon), Active(file), Inactive(file), Unevictable, Mlocked, SwapTotal, SwapFree, Dirty, Writeback, AnonPages, Mapped, Shmem, Slab, SReclaimable, SUnreclaim, KernelStack, PageTables, NFS_Unstable, Bounce, WritebackTmp, CommitLimit, Committed_AS, VmallocTotal, VmallocUsed, VmallocChunk, HardwareCorrupted, DirectMap4k, DirectMap2M, loadavg:: 12/10/2012 12:13:44, 3, 0, 3340, 380736, 258488, 4470692, 0, 0, 0, 48, 604, 876, 4, 1, 95, 0, 0, 8404788, 380736, 258488, 4256568, 20, 4088444, 3341192, 2377752, 535668, 1710692, 2805524, 0, 0, 2103292, 2099952, 576, 0, 2912980, 60420, 572, 239008, 214104, 24904, 3944, :::A8EEE9FC-42BA-11E2-AAA1-E8F86A583D9E:::
    Dec 10 12:13:44 vvx-mail-01 zimbramon[6947]: :::A8EEE9FC-42BA-11E2-AAA1-E8F86A583D9E:::0, 0, 0, 0, 6305684, 4828440, 34359738367, 5196, 34359731872, 0, 8388608, 0, 0.47
    
    zmsoap -z GetLoggerStatsRequest stats/@name=zmmtastats | head -20
    <GetLoggerStatsResponse xmlns="urn:zimbraAdmin"/>
    
    zmsoap -z GetLoggerStatsRequest stats/@name=zmmtastats @limit=1 | head -50
    <GetLoggerStatsResponse xmlns="urn:zimbraAdmin"/>

  5. #5
    stophi is offline Active Member
    Join Date
    Feb 2011
    Posts
    27
    Rep Power
    4

    Default

    ooooooooooooooohhh - after restarting zmloggerctl now also the services in admin-interface are shown green again :-)
    but I am wondering why /opt/zimbra/libexec/zmrrdfetch is not working.....
    so zimbra is doing the syslog-ng.conf not correct?!

  6. #6
    stophi is offline Active Member
    Join Date
    Feb 2011
    Posts
    27
    Rep Power
    4

    Default

    bill what do you think about it?

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Server stats not working
    By tecnoway in forum Installation
    Replies: 0
    Last Post: 06-16-2011, 08:45 PM
  2. Stats not working on ZCS 6.0
    By sherabi in forum Administrators
    Replies: 8
    Last Post: 01-26-2011, 01:38 PM
  3. Server stats not working
    By furlongm in forum Installation
    Replies: 0
    Last Post: 05-24-2010, 11:35 PM
  4. Stats not working in the 4.5.7
    By Max Ma in forum Administrators
    Replies: 1
    Last Post: 09-24-2007, 11:20 PM
  5. Stats Not Working
    By dinux in forum Installation
    Replies: 3
    Last Post: 09-14-2007, 11:43 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •