zmlogger and mysql... performance issues...

[tnelson]

Hello fellow Zimbra users/admins... I've been experiencing some very high CPU loads on our server. The load is generally caused by the zmlogger process and in turn, mysql. Upon disabling the zmlogprocess cron job as specified in another thread, the load has improved considerably. However, at times, the load still spikes and the system becomes slow. Upon closer inspection, I'm seeing that zmlogger/mysql are killing the machine again. Looking in the /opt/zimbra/logger/db/data/zimbra_logger dir, I'm also seeing HUGE databases such as these:

-rw-rw---- 1 zimbra zimbra  12G Jul 25 12:40 raw_logs.MYD
-rw-rw---- 1 zimbra zimbra 3.3G Jul 25 12:40 raw_logs.MYI

Our server runs an Intel Core 2 Duo@1.6ghz, 2GB RAM, 2X80GB SATAII drives in RAID1, and about 20 users total with usually around 5 concurrent users accessing using the full AJAX web interface. The platform is CentOS5 running Zimbra 5.0.4_GA_2101.RHEL5

Are my system loads incredibly high due to the massive size of the databases? Shouldn't there be a rotation or retention setting for how long to keep these "raw logs"? What can be done to improve the performance of zmlogger/mysql?

All suggestions greatly appreciated. Thank you!

[tnelson]

Any word? I have to believe I'm not the only one having these problems...

[ashbc]

Hi,

I seem to be having the same problem here (seeing loads upwards to 250!! before deciding to reboot).
After further inspection, I've found that shutting off zmlogger & zmswatch seems to reduce load dramaticly.

as user "zimbra"

# zmloggerctl stop ;
# zmswatchctl stop ;

Now, personaly I don't find this to be a "fix" - just a temporary relief - as you loose all the graphing charts in the Control Panel (:7071) once you do this. But none the less, it dropped my load enough to make the server usable once again (down to 0.5 - sometimes climbing to 1.5).

I've setup many mail servers in the past ... from high to low traffic Volume ... Zimbra is nice with its webmail-features, but rather heavy on the load ... i totaly suspect the logging system causing this ... Compared to a Postfix/Mysql/[Your Choice Of Webmail] setup ... Zimbra scoes 10/10 for features ... but 3/10 for performance when having lots of Webmail activity.

Is Zimbra looking into this performance issue? i am seeing questions about this all over the forum.

Cheers,

Ash

[tnelson]

I have yet to find an absolute fix for this. I've simply disabled the zmlog processes in the zimbra user crontab. On occasion though, the process still starts and sends the load on my servers to 10+. I simply notice the performance hit, log in, then killall the processes. I'm interested in a 'real' fix for this as well. It seems to stem from the log processes raping the mysql database. On my systems, these databases have grown to massive proportions (think 30GB+) and at some point became corrupt. Even after a proper 'myisamchk -r' on them, the load never gets and better and the DB is never pruned of old entries. This problem alone is the only thing "wrong" with Zimbra that I can see.

[dazza_act]

After upgrading to version 6.0 I've had to disable zmlogger as well due to it taking about 95% of the cpu about once per day.

How should I go about troubleshooting this issue?

[Luke13172]

I would also like to see a fix for this.

It took me a long time to talk my boss into this server and upgrading us from an OLD and outdated MDaemon Server to this beast. We ordered this machine configured as pure overkill for what it was intended for I do not want to have to talk my boss into another server buy for a long time. The only thing I want to have to talk my boss into is three more additional hard drives for the thing it can hold eight.

With that said even with Four Six core processors humming happily away I see loads as high as 4.5 avg hovers right around .3 normally but when that damm zmlogger kicks in it kicks the fans in that machine into high gear and I go look and it's being slammed by zmlogger there's nothing else running on this server other than SSH.

zmlogger =

I even partitioned the hard drive space according to the Zimbra installation walk thru I found online on best partitioning for Zimbra.

I'm pretty sure our machine is more than powerful enuff for Zimbra like I said I went extreme overkill for what Zimbra would actually require.

Even with 4.5 load it doesn't make the system unusable only real noticeable lag comes from pop3 access at least that is the only slow down I noticed when it's under high load.

Side Note:
java seems to run right at 2gb memory
mysql seems to run right at 1gb memory

One of my other administrators said she saw java climb to 4gb the other night while mysql stay around 1gb is that normal ?

We are running

Dell PowerEdge R900
Four Six Core X7460 Xeon, 2.67GHz, 16M Cache, 1066Mhz FSB
32GB Memory, 8X4GB, 667MHz
PERC RAID Controller, RAID 0
Five 73GB 10K RPM Serial-Attach SCSI 3Gbps 2.5-in HotPlug Hard Drive's
Four Fully Integrated Broadcom® 5708 Gigabit NICs, TOE Capable
Two External USB Western Digital 2TB Hard Drives for Backups

OS
Ubuntu 8.04 LTS (64bit version)

Zimbra
Release 6.0.0_GA_1802.UBUNTU8_64 UBUNTU8_64 FOSS

Dev's please fix this asap..

[FritzBrause]

Originally Posted by Luke13172

I would also like to see a fix for this.

Zimbra
Release 6.0.0_GA_1802.UBUNTU8_64 UBUNTU8_64 FOSS

Dev's please fix this asap..

... for 6.0.0 GA I found a workaround. Have a look at this thread
http://www.zimbra.com/forums/install...igh-cpu-2.html

Regards