High CPU usage on server and client

[jack-Z]

Hi,

I am doing a test drive with Zimbra version 3.10.GA_332.FC4 on a PC with 2GB RAM and 2.6GHZ P4 CPU. There are just less than 10 users created and I found that CPU usage was very high from time to time, even though there is only one user trying to connect through web at the moment. Besides, on the client side we noticed the CPU usage hit 100% when trying to connect to Zimbra, we tried both firefox and IE on Windows XP.

This box is dedicated for Zimbra only, I installed the Zimbra by following the instructions provided, nothing else not mentioned in the instructions is installed.

Below is the "top" result I captured:

[root@mail ~]# top
top - 11:39:17 up 30 days, 13:02, 1 user, load average: 0.14, 0.15, 0.16
Tasks: 111 total, 3 running, 108 sleeping, 0 stopped, 0 zombie
Cpu(s): 26.2% us, 3.3% sy, 0.0% ni, 70.4% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 2067660k total, 1988456k used, 79204k free, 78240k buffers
Swap: 4192956k total, 0k used, 4192956k free, 1189664k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12039 zimbra 19 0 461m 22m 10m R 28.3 1.1 0:00.85 java
13972 zimbra 22 0 873m 245m 14m S 1.0 12.1 4:31.07 java
14024 zimbra 16 0 737m 57m 3632 S 0.3 2.8 11:25.69 mysqld
9745 root 16 0 2024 1008 784 R 0.3 0.0 0:01.24 top
1 root 16 0 1744 568 492 S 0.0 0.0 0:01.57 init
2 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
3 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
4 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/0
5 root 12 -5 0 0 0 S 0.0 0.0 0:00.01 khelper
6 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kthread
8 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid
80 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/0
83 root 15 0 0 0 0 S 0.0 0.0 0:00.00 khubd
129 root 20 0 0 0 0 S 0.0 0.0 0:00.00 pdflush
130 root 15 0 0 0 0 S 0.0 0.0 0:01.67 pdflush


Any suggestion is highly appreciated.

[phoenix]

First, you could try upgrading to the most recent release of Zimbra. High CPU usage can be caused by the logger, can you disable/stop that and see if the CPU usage goes down.

[jack-Z]

Just upgraded to the latest version - 3.13_GA_474.FC4, but still got high CPU usage:

top - 23:07:05 up 32 days, 30 min, 1 user, load average: 0.25, 0.55, 0.51
Tasks: 111 total, 2 running, 109 sleeping, 0 stopped, 0 zombie
Cpu(s): 47.0% us, 5.3% sy, 0.0% ni, 47.7% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 2067660k total, 1857432k used, 210228k free, 103480k buffers
Swap: 4192956k total, 60k used, 4192896k free, 1234908k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19885 zimbra 17 0 43248 8832 2972 S 0.3 0.4 0:00.18 slapd
24382 zimbra 16 0 7060 3324 1736 S 0.3 0.2 0:00.10 zmmtaconfig
1 root 16 0 1744 568 492 S 0.0 0.0 0:01.59 init
2 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
3 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
4 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/0


Any suggestions?

Thanks!

[jack-Z]

Here is another "top" capture:

[root@mail zcs]# top
top - 23:20:07 up 32 days, 43 min, 1 user, load average: 0.41, 0.25, 0.33
Tasks: 122 total, 5 running, 117 sleeping, 0 stopped, 0 zombie
Cpu(s): 81.6% us, 18.4% sy, 0.0% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 2067660k total, 1933072k used, 134588k free, 103528k buffers
Swap: 4192956k total, 60k used, 4192896k free, 1236268k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
25514 zimbra 16 0 104m 15m 3128 S 79.2 0.8 0:09.42 mysqld
3472 zimbra 18 0 7288 4592 2132 R 19.6 0.2 0:00.89 zmlogprocess
25988 zimbra 16 0 736m 53m 3356 S 0.7 2.6 0:00.69 mysqld
26437 zimbra 16 0 866m 202m 15m S 0.7 10.0 0:11.82 java
3458 root 16 0 2024 1012 784 R 0.3 0.0 0:00.02 top
1 root 16 0 1744 568 492 S 0.0 0.0 0:01.59 init
2 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
3 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0

[phoenix]

Does this machine you're using as the zimbra server have hyperthreading? If it does adnd it's active try disabling it, you could also try disabling the logger as I mentioned in my last post. Try those test individually and see if you can narrow down the cause.

[jack-Z]

How to disable the logger? In admin concole I de-select the logger service, and restart the server by running "zmcontrol shutdown" and "zmcontrol start", is there anything else I need to do?

Still notice high CPU usage, and most of them are from "java" as below:

[zimbra@mail root]$ top
top - 11:52:20 up 32 days, 13:15, 1 user, load average: 0.23, 0.26, 0.19
Tasks: 112 total, 3 running, 109 sleeping, 0 stopped, 0 zombie
Cpu(s): 67.4% us, 6.6% sy, 0.0% ni, 25.6% id, 0.0% wa, 0.3% hi, 0.0% si
Mem: 2067660k total, 1820988k used, 246672k free, 107904k buffers
Swap: 4192956k total, 856k used, 4192100k free, 1212856k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5184 zimbra 25 0 461m 22m 10m R 27.6 1.1 0:00.83 java
2835 zimbra 17 0 7072 3324 1736 S 0.3 0.2 0:00.09 zmmtaconfig
4137 zimbra 16 0 2024 1008 784 R 0.3 0.0 0:00.49 top
1 root 16 0 1744 568 492 S 0.0 0.0 0:01.59 init

Will check what I can do with the CPU later.

[KevinH]

You can just comment out the zmlogprocess cron job for starters.

[jack-Z]

HI,

I disabled the hyperthreading in CMOS, and disabled the logger service in crontab, but still get the high CPU usage.
Can you advise what else I can do?

Thanks,

Jack

Here is a capture of high CPU usage:

top - 12:38:24 up 10 min, 2 users, load average: 0.13, 0.24, 0.18
Tasks: 105 total, 3 running, 102 sleeping, 0 stopped, 0 zombie
Cpu(s): 65.8% us, 8.0% sy, 0.0% ni, 25.6% id, 0.7% wa, 0.0% hi, 0.0% si
Mem: 2067660k total, 552744k used, 1514916k free, 16464k buffers
Swap: 4192956k total, 0k used, 4192956k free, 175348k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4510 zimbra 25 0 461m 18m 9436 R 19.3 0.9 0:00.58 java
2868 zimbra 17 0 7072 3324 1736 S 0.7 0.2 0:00.09 zmmtaconfig
2794 zimbra 18 0 56320 8624 2900 S 0.3 0.4 0:00.10 slapd
1 root 16 0 1748 572 492 S 0.0 0.0 0:01.34 init
2 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
3 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
4 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/0
5 root 12 -5 0 0 0 S 0.0 0.0 0:00.01 khelper
6 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kthread
8 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid
80 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/0
83 root 15 0 0 0 0 S 0.0 0.0 0:00.00 khubd
129 root 20 0 0 0 0 S 0.0 0.0 0:00.00 pdflush
130 root 15 0 0 0 0 S 0.0 0.0 0:00.00 pdflush

Here is my crontab table:

[zimbra@mail root]$ crontab -l
# ZIMBRASTART -- DO NOT EDIT ANYTHING BETWEEN THIS LINE AND ZIMBRAEND
#
# Log pruning
#
30 2 * * * find /opt/zimbra/log/ -type f -name \*log\* -mtime +8 -exec rm {} \; > /dev/null 2>&1
#
# Status logging
#
#*/2 * * * * /opt/zimbra/libexec/zmstatuslog
#*/10 * * * * /opt/zimbra/libexec/zmdisklog
#
# crontab.ldap
#
#
# Spam training cleanup
#
45 23 * * * /opt/zimbra/bin/zmtrainsa --cleanup >> /opt/zimbra/log/spamtrain.log 2>&1
#
# crontab.store
#
# Log pruning
#
30 2 * * * find /opt/zimbra/tomcat/logs/ -type f -name \*log\* -mtime +8 -exec rm {} \; > /dev/null 2>&1
0 2 * * * /opt/zimbra/libexec/zmroll_catalina > /dev/null 2>&1
#
# Purge messages
#
0 3 * * * /opt/zimbra/libexec/zmpurgemessages > /dev/null 2>&1
#
# Clean up the quarantine dir
#
0 1 * * * find /opt/zimbra/amavisd/quarantine -type f -mtime +7 -exec rm -f {} \; > /dev/null 2>&1
#
# Table maintenance
#
30 1 * * * /opt/zimbra/libexec/zmmaintaintables >> /dev/null 2>&1
# BACKUP BEGIN
# BACKUP END
#
# crontab.logger
#
# process logs
#
#00,10,20,30,40,50 * * * * /opt/zimbra/libexec/zmlogprocess > /tmp/logprocess.out 2>&1
#
# Graph generation
#
10 * * * * /opt/zimbra/libexec/zmgengraphs >> /tmp/gengraphs.out 2>&1
#
# Daily reports
#
10 1 * * * /opt/zimbra/libexec/zmdailyreport | /opt/zimbra/postfix/sbin/sendmail root
#
# crontab.mta
#
#
# Queue logging
#
*/10 * * * * /opt/zimbra/libexec/zmqueuelog
#
# Spam training
#
0 23 * * * /opt/zimbra/bin/zmtrainsa >> /opt/zimbra/log/spamtrain.log 2>&1
# ZIMBRAEND -- DO NOT EDIT ANYTHING BETWEEN THIS LINE AND ZIMBRASTART


Here is the CPU info:
[zimbra@mail root]$ exit
exit
[root@mail ~]# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) Pentium(R) 4 CPU 2.60GHz
stepping : 9
cpu MHz : 2593.740
cache size : 512 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid xtpr
bogomips : 5144.57


And here is the MEM usage:

[root@mail ~]# free -m
total used free shared buffers cached
Mem: 2019 531 1487 0 16 171
-/+ buffers/cache: 343 1676
Swap: 4094 0 4094

[layer3guru]

In my testing I ran the latest version of zimbra on a p4 2.8ghz with 2gig of mem on an IBM server. With no users at all created and the box processing no mail. My monitoring box which I use cacti reported the Zimbra server running at 20% cpu utilization all the time this box is not on the network processing any mail at all. I am not bashing this is just a FYI for anyone that may come along and find this thread. I wil lsay this did not deter me from deploying in a minimal testing environment on a p4 3.2ghz with 2gig of mem and 10 users basically all of the sysadmins in my group are putting it through the paces. And now that we have these 10 users on the 3.2ghz box cpu is about the same 20% utilization but a few of the sysadmins get upwards of 1500 emails a day from different mailinglist etc.

[Billa]

just come across this thread while googling. I have seen such issues with too many client complaining about High CPU usage cause of tomcat service.

I would suggest to run tomcat by keeping one CPU away from it. Like if you have quad core CPU I mean if you have 4 cpus on your machine then you can modify the tomcat instance to run it on 3cpus only so that you can have at least the access to server to moniter whats going wrong.

Here is the way how you can solve the problem of "Tomcat Hight CPU usage problem"