Zimbramon disk warning

[pcad]

I am getting floods of disk warning alerts like so:
Dec 25 19:10:00 zimbra zimbramon[11651]: 11651:crit: Disk warning: zimbra.xxxxx.xxx: /dev/cciss/c0d2p1 at 95%

My backup disk became full leading up to the end of December. This was also the same time we upgraded to NE 6.0.9. I subsequently cleared space on my backup drive which is a separate mount point than /opt/zimbra. Since then, I have received three floods of the Disk warning messages like the one above. These are not new alerts, these are the same alerts that occurred just before the holidays as you can see in the above alert.

A check with df -h yields:

Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p3      59G   12G   44G  21% /
/dev/cciss/c0d1p1     271G   95G  163G  37% /opt
/dev/cciss/c0d2p1     271G   68G  190G  27% /zimbra-data
/dev/cciss/c0d0p1      99M   23M   71M  25% /boot
tmpfs                 3.9G     0  3.9G   0% /dev/shm
/dev/sdb1             1.8T  1.2T  573G  68% /backup2

I prefer not to change or comment out the thresholds of my alerts,.
Is this a bug? I've read some bugzilla entries. I have not attached a thumb drive when the floods have occurred. The above drive is my backup drive.

Why am I getting multiple floods of dated alerts? We are talking thousands and thousands of messages which kicks up the load average very high.

Thanks for any help you might have.
Hylon

[lytledd]

Is it possible that you have lots of deferred mail sitting out in the Zimbra queue?

[pcad]

Hi lytledd,
No, the floods of email are actually duplicates from an earlier time. They go right to the active queue, whereupon I delete them before they are sent out. At any one time, I have between 5-10 deferred emails. When the flood comes, we are talking like 400-500 in the active queue.
Thanks for looking,

[pcad]

This is happening again to our server.
Load average is 8 and above 400-600 messages going right to active queue.

None of my disks are close to the threshold for capacity.

No usb thumb drives have been inserted.

Can anyone think of anything else to check.

[lytledd]

What does your /opt/log/mailbox.log show as the first entry?

Does /var/log/zimbra.log give any clues?

Or even

/var/log/mail.log mail.err or even mail.warn

Can you post 1 of the mails, including any headers?

Doug

[pcad]

Originally Posted by lytledd

What does your /opt/log/mailbox.log show as the first entry?

The first entry is a zimbraweblient connection.

Does /var/log/zimbra.log give any clues?



Or even

/var/log/mail.log mail.err or even mail.warn

Can you post 1 of the mails, including any headers?

Return-Path: zimbra@zimbra.xxxxx.edu
Received: from zimbra.xxxxx.edu (LHLO zimbra.xxxxx.edu) (66.146.237.151) by
zimbra.xxxxx.edu with LMTP; Thu, 24 Feb 2011 12:47:16 -0500 (EST)
Received: from localhost (localhost.localdomain [127.0.0.1])
by zimbra.xxxxx.edu (Postfix) with ESMTP id CB48A3D2002B
for <admin@xxxxx.edu>; Thu, 24 Feb 2011 12:47:03 -0500 (EST)
X-Virus-Scanned: amavisd-new at zimbra.xxxxx.edu
X-Spam-Flag: NO
X-Spam-Score: -1.901
X-Spam-Level:
X-Spam-Status: No, score=-1.901 tagged_above=-10 required=6
tests=[BAYES_00=-1.9, NO_RELAYS=-0.001] autolearn=ham
Received: from zimbra.xxxxx.edu ([127.0.0.1])
by localhost (zimbra.xxxxx.edu [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id JST8u-kmPcyP for <admin@xxxxx.edu>;
Thu, 24 Feb 2011 12:47:02 -0500 (EST)
Received: by zimbra.xxxxx.edu (Postfix, from userid 501)
id 076063D2002D; Thu, 24 Feb 2011 12:47:01 -0500 (EST)
To: admin@xxxxx.edu
From: admin@xxxxx.edu
Subject: Disk /dev/cciss/c0d2p1 at 85% on zimbra.xxxxx.edu:
Message-Id: <20110224174702.076063D2002D@zimbra.xxxxx.edu>
Date: Thu, 24 Feb 2011 12:47:01 -0500 (EST)

Dec 24 06:20:06 zimbra zimbramon[1451]: 1451:err: Disk warning: zimbra.xxxxx.edu: /dev/cciss/c0d2p1 at 85%

Doug

Again, the error messages date back to when the errors took place, December 24th.

So it seems, the errors are queued, and for whatever reason, the messages are re-released.

I am checking through the other logs you mention. I am not seeing any related to errors as of yet, but continueing to look.

Thanks for thinking about this.

[pcad]

Hello lytledd,
I looked through the logs you mentioned and haven't seen anything out of the ordinary.

The flood event happened over the weekend. The same pattern, a set of disk warning errors in addition to a couple events related to services starting and stopping between the dates of Dec 23rd and Jan 25th.

Somewhere there is a queue of errors that are reset to the admin account.

Thanks for giving this a look.

Hylon

[lytledd]

Originally Posted by pcad

Return-Path: zimbra@zimbra.xxxxx.edu
Received: from zimbra.xxxxx.edu (LHLO zimbra.xxxxx.edu) (66.146.237.151) by
zimbra.xxxxx.edu with LMTP; Thu, 24 Feb 2011 12:47:16 -0500 (EST)

Is the above IP address the IP of the receiving Zimbra server? If not, then you've got another server sending you those messages.

Doug

[pcad]

Yes, that would be the correct server.

[pcad]

Upgraded to NE 6012 and the floods of recycled and old disk warning messages have ceased.