ZCS Administrator's Guide, Open Source Edition 4.5 Rev 2 11/07
Table of Contents Previous Next Index


Monitoring Zimbra Servers

Monitoring Zimbra Servers
The Zimbra Collaboration Suite includes the following to help you monitor the Zimbra servers, usage, and mail flow.
Zimbra Logger package to capture and display server statistics and server status, for message tracing, and to create nightly reports
Also, selected error messages generate SNMP traps, which can be monitored using a SNMP tool.
Note: Checking the overall health of the system as a whole is beyond the scope of this document.
Zimbra Logger
Zimbra-Logger includes tools for syslog aggregation, reporting, and message tracing. Installing the Logger package is optional, but if you do not install Logger, Server Statistics and Server Status information is not captured and message tracing is not available.
In environments with more than one Zimbra server, Logger is enabled on only one mailbox server. This server is designated as the monitor host. The Zimbra monitor host is responsible for checking the status of all the other Zimbra servers and presenting this information on the Zimbra administration console. The information updates every 10 minutes.
Note: In a multi-server installation, you must set up the syslog configuration files on each server to enable logger to display the server statistics on the administration console, and you must enable the logger host. If you did not configure this when you installed ZCS, do so now.
To enable Server Statistics.
1.
On each server, as root, type /opt/zimbra/bin/zmsyslogsetup. This enables the server to display statistics.
2.
On the logger monitor host, you must enable syslog to log statistics from remote machines.
a.
Edit the /etc/sysconfig/syslog file, add -r to the SYSLOGD_OPTIONS setting, SYSLOGD_options=”-r -m 0”
b.
Stop the syslog daemon. Type /etc/init.d/syslogd stop.
c.
Start the syslog daemon. Type /etc/init.d/syslogd start.
These steps are not necessary for a single-node installation.
Reviewing Server Status
The Server Status page lists all servers and services, their status, and when the server status was last checked. The servers include the MTA, LDAP, and mailbox server. The services include MTA, LDAP, Mailbox, SNMP, Anti-Spam, Anti-Virus, Spell checker, and Logger.
To start a server if it is not running, use the zmcontrol CLI command. You can stop and start services from the administration console, Servers>Services tab.
Server Performance Statistics
The Server Statistics page shows bar graphs of the Message Count, Message Volume, Anti-Spam, and Anti-Virus activity. The information is displayed for the last 48 hours, and 30, 60, and 365 days.
Message Count displays the number of messages sent and received per hour and per day.
Message Volume displays the aggregate size in bytes of messages sent and receive per hour and per day.
Anti-Spam/Anti-Virus Activity displays the number of messages that were checked for spam or viruses and the number of messages that were tagged as spam or deemed to contain a virus.
Disk displays the disk usage and the disk space available for individual servers. The information is displayed for the last hour, day, month and year.
The Message Count and the Anti-spam/Anti-virus Activity graphs display a different message count because:
Outbound messages may not go through the Amavisd filter, as the system architecture might not require outbound messages to be checked.
Message are received and checked by Amavisd for spam and viruses before being delivered to all recipients in the message. The message count shows the number of recipients who received messages.
Tracing Messages
You can trace an email message that was sent or received within the last 30 days.
Each email message includes a header that shows the path of an email from its origin to destination. This information is used to trace a message’s route when there is a problem with the message.
The CLI utility, zmmsgtrace is run to find email messages by the follow:
Message ID, -i [msd_id]
Sender address (From), -s [sender_addr]
IP Address sent from, -f [ip_address]
Date and time, -t yyyymmdd(hhmmss)
The Zimbra email message header can be viewed from the Zimbra Web Client Message view. Right-click on a message and select Show Original.
Note: If messages are viewed by Conversation view, first open the conversation to view the messages. Then select the message.
Examples
Note: Message trace is run from the Zimbra monitor host, which is the server where Logger is enabled.
zmmsgtrace -i 3836172.14011130514432170
zmmsgtrace -s user@example.com -r user2@example2.com -t 20051105
The following message trace example was looking for messages sent from sender, jdoe to recipient address, aol.com any time within the last 30 days. The details show that two messages were sent, and it shows to whom the messages were sent.
 
$ zmmsgtrace -s jdoe -r aol.com
Tracing messages
from jdoe
to aol.com


Message ID 17357409.1128717619728.JavaMail.companya@example.com
jdoe@example.com -->
kumsh@aol.com
Recipient kumsh@aol.com
2005-01-07 13:40:19 - example.com (10.10.000.20) -->
2005-01-07 13:40:20 - example --> 000.0.0.1 (100.0.0.0) status sent
2005-01-07 13:40:20 Passed by amavisd on example (CLEAN) HITS: -5.773 in 539 ms
2005-01-07 13:40:20 - localhost.localdomain (100.0.0.1) --> example
2005-01-07 13:40:20 - example --> mta02.example.com (0.00.000.00) status sent

Message ID 3836172.14011130514432170.JavaMail.root@example.com
jdoe@example.com -->
harma@aol.com
lt@hotmail.com
Recipient harma@aol.com
2005-01-28 08:47:13 - localhost.localdomain (000.0.0.1) --> example
2005-01-28 08:47:13 - example --> mta02.example.com ( 0.70.000.09) status sent

2 messages found
Generating Daily Mail Reports
When the Logger package is installed, a daily mail report is automatically scheduled in the crontab. The Zimbra daily mail report includes the following information:
The report runs every morning at 4 a.m. and is sent to the administrator’s email address.
You can configure the number of accounts to include in the report. The default is 50 sender and 50 recipient accounts.
To change the number of recipients to add to the report, type:
zmlocalconfig -e zimbra_mtareport_max_recipients=<number>
To change the number of senders to add to the report, type:
zmlocalconfig -e zimbra_mtareport_max_senders=<number>
Monitoring Mailbox Queues
If you are having problems with mail delivery, you can view the mail queues from the administration console, Monitoring Mail Queues page to see if you can fix the mail delivery problem. When you open Mail Queues, the content of the Deferred, Incoming, Active, Hold, and Corrupt queues at that point in time can be viewed. You can view the number of messages and where they are coming from and going to. For description of these queues, see "Zimbra MTA Message Queues” .
Figure 8: Mailbox Queue Page
For each queue, the Summary pane shows a summary of messages by receiver domain, origin IP, sender domain, receiver address, sender address, and for the Deferred queue, by error type. You can select any of the summaries to see detailed envelope information by message in the Messages pane.
The Messages pane displays individual message envelope information for search filters selected from the Summary pane.
The the following Mailbox Queue functions can be performed for all the messages in a queue:
Hold to move all messages in the queue being viewed to the Hold queue. Messages stay in this queue until the administrator moves them.
Release to remove all message from the Hold queue. Messages are moved to the Deferred queue.
Requeue requeues all messages in the queue being viewed. Requeuing messages can be used to send messages that were deferred because of a configuration problem that has been fixed. Messages are re-evaluated and earlier penalties are forgotten.
Delete deletes all messages in the queue being viewed.
The Zimbra MTA, Postfix queue file IDs are reused. If you requeue or delete a message, note the message envelope information, not the queue ID. It is possible that when you refresh the mail queues, the queue ID could be used on a different message.
Flushing the Queues
In addition to moving individual messages in a specific queue, you can flush the server. When you click the Flush button, delivery is immediately attempted for all messages in the Deferred, Incoming and Active queues.
Monitoring Mailbox Quotas
You can view the mailbox quota for all accounts from the administration console in Monitoring>Server Statistics>Mailbox Quota tab. The Mailbox Quota tab gives you an instant view of the following information for each account:
When an account quota is reached all mail messages are rejected. Users will need to delete mail off the server to get below their quota limit, or you can increase their quota.
Log Files
The Zimbra Collaboration Suite logs its activities and errors to a combination of system logs through the syslog daemon as well as Zimbra specific logs on the local file system. The logs described below are the primary logs that are used for analysis and troubleshooting.
Local logs contain ZCS activity are in the /opt/zimbra/log directory.
audit.log. This log contains authentication activity of users and administrators, as well as mail delivery, login failure, and Tomcat start/stop activities.
clamd.log. This log contains activity from the antivirus application clamd.
freshclam.log. This log contains log information related to the updating of the clamd virus definitions.
logger_myslow.log. This slow query log consists of all SQL statements that took more then long_query_time seconds to execute. Note: long_query_time is defined in /opt/zimbra/my.logger.cnf.
mailbox.log. This log is a Tomcat log4j server log containing the logs from the mailbox server. This includes the mailbox store, LMTP server, IMAP and POP servers, and Index server. (Note: prior to ZCS 4.5, this log was called /opt/zimbra/log/zimbra.log.)
myslow.log. This slow query log consists of all SQL statements from the mailbox server that took more then long_query_time seconds to execute. Note: long_query_time is defined in /opt/zimbra/my.cnf.
spamtrain.log. This log contains output from zmtrainsa during regularly scheduled executions from the cron.
sync.log. This log contains information about Zimbra mobile sync operations.
Other logs include:
/opt/zimbra/tomcat/logs/catalina.out. This is where Tomcat-specific activity is logged.
/opt/zimbra/db/data. <hostname>.err. This is the message store database error log.
/opt/zimbra/logger/db/data. <hostname>.err. This is the Logger database error log.
ZCS activity logged to System syslog system
/var/log/zimbra.log. The Zimbra syslog details the activities of the ZCS MTA (Postfix, amavisd, antispam, antivirus), Logger, Authentication (cyrus-sasl), and Directory (OpenLDAP),
Syslog file is written by the operating system, and contains a subset of the messages written to the local logs. SNMP monitoring typically looks at the syslog file and generates traps for critical errors.
Using log4j to Configure Logging
The Zimbra server uses log4j, a Java logging package as the log manager. By default, the Zimbra server has log4j configured to log to the local file system. You can configure log4j to direct output to another location. Go to the Log4j Website for information about using log4j.
Logging Levels
The levels for Zimbra logging messages are shown below, along with which ones generate SNMP traps and where, by default, each message type is logged.
Table 1 Zimbra Logging Levels
SNMP Trap?
Single user error, unexpected; for example, can’t open index
SNMP
SNMP Monitoring Tools
You will probably want to implement server monitoring software in order to monitor system logs, CPU and disk usage, and other runtime information.
Zimbra uses swatch to watch the syslog output to generate SNMP traps.
SNMP Configuration
Zimbra includes an installer package with SNMP monitoring. This package should be run on every server (Zimbra, OpenLDAP, and Postfix) that is part of the Zimbra configuration.
The only SNMP configuration is the destination host to which traps should be sent.
Errors Generating SNMP Traps
The Zimbra error message generates SNMP traps when a service is stopped or is started. You can capture these messages using third-party SNMP monitoring software and direct selected messages to a pager or other alert system.
Checking MySQL
Beginning with 4.5.7, the MySQL database is automatically checked weekly to verify the health of the database. This check takes about an hour. If any errors are found, a report is sent to the administrator’s account. The report name that runs the MySQL check is zmbintegrityreport, and the crontab is automatically configured to run this report once a week.
Note: When the MySQL database is checked, running this report can consume a significant amount of I/O. This should not present a problem, but if you find that running this report does affect your operation, you can change the frequency with which zmbintegrityreport is run. See the Appendix C Zimbra Crontab Jobs.
 

Monitoring Zimbra Servers

ZCS Administrator's Guide, Open Source Edition 4.5 Rev 2 11/07
Copyright © 2007 Zimbra Inc.