Daily Mail Report - missing data

[klinet]

My daily mail report seems to be working only in the afternoon.

The per hour traffic summary does not begin until 2PM.

See attachment

Anyone have any thoughts?

Thanks
Todd

[LMStone]

FWIW we are seeing the exact same thing, except the start-of-data-time for us is different.

We've been busy sorting out more pressing issues from our recent Zimbra upgrade (see ), so haven't gotten to this problem fully just yet since we only got stats back working properly yesterday.

So far, what we see is that the logs /var/log/zimbra*.log seem to be getting rotated around 6:15pm our time. Our server's hardware clocks are set to UTC, and our Time Zone on the servers (and in Zimbra) is set to Eastern, i.e. GMT -4 (daylight savings time).

We'll look at the logrotation scripts and confirm system time later and report back here whatever we find.

Curious what time zone your system is configured for and if the number of hours difference in our respective start-of-data-times in the daily mail report matches exactly with the hours difference between our two time zones.

Hope that helps,
Mark

[klinet]

Thanks for the info, I think you may be onto something with the log rotation times. I see that you are also using SLES 10x64. We are in Central time, so if you are seeing a one hour difference in the start of your hourly info then I think we have a good place to be looking. This is also a low priority for us, but still looking at it as time permits.

Thanks
Todd

[LMStone]

I think I have identified the root cause of this issue...

I looked at the time stamps for the /var/log/*.bz2 files on about a dozen SLES systems this morning (some Zimbra hosts, some not), and saw that the log files being rotated by scripts in /etc/cron.daily had time stamps of roughly on the hour and fifteen/thirty/forty-five minutes after the hour -- and that the hours were all pretty random. There was no correlation between time zones at all.

I also noticed that the time of day the files were rotated changed by an hour when Daylight Savings Time kicked in.

The new Zimbra logging system as I understand it, including the daily mail report and statistics in the Admin console, relies on data in /var/log/zimbra.log and /var/log/zimbra-stats.log being available to parse.

When these log files are rotated in the middle of the day, the data in the daily mail report and stats in the Admin Console is incomplete.

So why are these log files being rotated by scripts in /etc/cron.daily on such a seemingly random basis?

Because unless the "DAILY_TIME" parameter in /etc/sysconfig/cron is set to a specific time, cron.daily is run by default at the next 15 minute interval past the last boot time hour -- until the server is once again rebooted.

On all the SLES systems I looked at, "DAILY_TIME" was blank.

Consequently, /etc/cron.daily/zimbra is being executed each day based on the last boot time of the server.

This is not what Zimbra I believe intends so I have filed a bug report here: Bug 43847 – Incorrect Log File Rotation Method Causing Missing Data in Daily Admin Report and Admin Console Stats Pages

Would be interesting if admins using other distros could comment on their setups here as well...

Hope that helps,
Mark

[uxbod]

Mark,

on CentOS it is controller by /etc/crontab

02 4 * * * root run-parts /etc/cron.daily

[LMStone]

Originally Posted by uxbod

Mark,

on CentOS it is controller by /etc/crontab

02 4 * * * root run-parts /etc/cron.daily

Thanks for posting that!

So, are there time periods in your daily mail report that are missing data?

All the best,
Mark

[phoenix]

Originally Posted by LMStone

So, are there time periods in your daily mail report that are missing data?

Not on my CentOS server, there is received mail in all the time slots.

[LMStone]

Originally Posted by phoenix

Not on my CentOS server, there is received mail in all the time slots.

Well, we'll see what happens in two days then!

Thanks Bill,
Mark

P.S. Unsupported OS eh?

[phoenix]

Originally Posted by LMStone

P.S. Unsupported OS eh?

Of course but I don't get support for my installation anyway.

[jrefl5]

Like uxbod, and phoenix I don't have any problems, but then I'm also on CENTOS.

you might want to try to set a reasonable time in "DAILY_TIME".

the zmdailyreport attempts to open both the current zimbra.log, and zimbra.log.1 or zimbra.log.1.gz so it should be able to get the prior day.