CPU High Spikes around every 2 minutes ZSC 7.1.4

Printable View

Show 40 post(s) from this thread on one page

02-01-2012, 01:56 PM
mrcoffee11

2 Attachment(s)

CPU High Spikes around every 2 minutes ZSC 7.1.4

Hi guys,

On my Fedora 11 x64 system I got this problem since my upgrade a few hours ago from version ZCS 7.0.0 to 7.1.4. Can't seem to find a solution for these hight CPU spikes.

I also have the same problem on a newly installed server running Centos 6.2 with 2GBs of RAM and a fresh install of ZCS 7.1.4 and zero mailboxes. Both are VM's are running on VMware vSphere5.

I edited the crontab adjusting jobs that run every 2 minutes to a longer time period.

Also entered these commands which worked for older versions:

Code:

zmlocalconfig -e zmmtaconfig_interval=6000 zmprov mcf zimbraLogRawLifetime 7d zmprov mcf zimbraLogSummaryLifetime 30d

Still got these spikes every 2 minutes, sometimes 2:20 minutes. (see the graphs)

Any suggestions on what this is and how to increase the time interval?

Thnx,
Roderick

Fedora 11 : Release 7.1.4_GA_2555.F11_64_20120105094338 F11_64 FOSS edition.
Centos 6.2: Release 7.1.4_GA_2555.RHEL6_64_20120105094542 CentOSUNKNOWN_64 FOSS edition.
02-08-2012, 01:25 AM
mrcoffee11

Bump. There must be a Zimbra developper or Zimbra-Super-Gourou who can answer this question?
02-13-2012, 07:54 AM
mrcoffee11

2 Attachment(s)

Still can't find a solution/cause. Other people seem to have the same issue.
http://www.zimbra.com/forums/adminis...tml#post236475

I uploaded another graph to show the difference in CPU usage before the upgrade from 7.0.0 and almost two weeks ago to 7.1.4.

Grtz,
Roderick
02-14-2012, 03:29 AM
Yves Pires

run iostat 4 for 20 seconds and post the output

also

Quote:

top

Quote:

df -h
02-14-2012, 03:39 AM
mrcoffee11

iostat 4

Code:

Linux 2.6.30.10-105.2.23.fc11.x86_64 (ventoux.r71.nl) 02/14/2012 avg-cpu: %user %nice %system %iowait %steal %idle 13.79 0.00 1.28 0.28 0.00 84.65 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 3.81 39.32 99.36 81013164 204731706 sda1 0.00 0.00 0.00 2488 42 sda2 3.81 39.31 99.36 81010404 204731664 dm-0 11.95 36.18 91.96 74550170 189501680 dm-1 1.32 3.14 7.39 6459984 15229984 avg-cpu: %user %nice %system %iowait %steal %idle 95.25 0.00 4.75 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 0.00 0.00 0.00 0 0 sda1 0.00 0.00 0.00 0 0 sda2 0.00 0.00 0.00 0 0 dm-0 0.00 0.00 0.00 0 0 dm-1 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 97.00 0.00 3.00 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 2.50 90.00 14.00 360 56 sda1 0.00 0.00 0.00 0 0 sda2 2.50 90.00 14.00 360 56 dm-0 3.25 90.00 14.00 360 56 dm-1 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 97.49 0.00 2.51 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 0.00 0.00 0.00 0 0 sda1 0.00 0.00 0.00 0 0 sda2 0.00 0.00 0.00 0 0 dm-0 0.00 0.00 0.00 0 0 dm-1 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 98.50 0.00 1.50 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 3.25 0.00 64.00 0 256 sda1 0.00 0.00 0.00 0 0 sda2 3.25 0.00 64.00 0 256 dm-0 8.00 0.00 64.00 0 256 dm-1 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 98.25 0.00 1.75 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 2.25 38.00 34.00 152 136 sda1 0.00 0.00 0.00 0 0 sda2 2.25 38.00 34.00 152 136 dm-0 5.00 38.00 34.00 152 136 dm-1 0.00 0.00 0.00 0 0 avg-cpu: %user %nice %system %iowait %steal %idle 98.25 0.00 1.75 0.00 0.00 0.00 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 29.00 0.00 334.00 0 1336 sda1 0.00 0.00 0.00 0 0 sda2 29.00 0.00 334.00 0 1336 dm-0 41.75 0.00 334.00 0 1336 dm-1 0.00 0.00 0.00 0 0

df -h /

Code:

Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_00-lv_root 20G 7.4G 11G 41% /

top

Code:

top - 12:39:09 up 23 days, 20:37, 1 user, load average: 1.53, 0.96, 0.86 Tasks: 105 total, 2 running, 103 sleeping, 0 stopped, 0 zombie Cpu(s): 88.1%us, 11.9%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 1588352k total, 1463056k used, 125296k free, 84388k buffers Swap: 511992k total, 81060k used, 430932k free, 323620k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 25163 zimbra 20 0 984m 165m 12m S 80.4 10.7 0:27.52 java 16097 zimbra 20 0 1517m 398m 11m S 0.3 25.7 1:21.16 java
02-14-2012, 10:50 AM
Yves Pires

maybe status log updating every 2min and consuming alot cpu %

try:

Quote:

su – zimbra

crontab -e

*/60 * * * * /opt/zimbra/libexec/zmstatuslog
02-14-2012, 11:39 AM
mrcoffee11

I already adjusted most of the crontab jobs directly after the upgrade, including this one. Thnx for trying. More sugestions welcome!
06-18-2012, 07:06 AM
mrcoffee11

Bump!
I still got this issue. Even a fresh install of Zimbra with NO USERS had the same CPU spikes of this JAVA process. I tried all known tricks:
- editing the crontab of user zimbra
- things like:
zmlocalconfig -e zmmtaconfig_interval=6000
zmprov mcf zimbraLogRawLifetime 7d
zmprov mcf zimbraLogSummaryLifetime 30d

Suggestions are more then appreciated!
06-18-2012, 08:58 AM
mrcoffee11

Upgraded to 7.2. Same issue!
Every 2 minutes and 20 seconds (140 seconds) a java proces takes up 90-99% of CPU capacity for about 35 seconds. I think it's not a cron job since the interval is around 140 seconds.

I noted the time interval:
17:48:10
17:50:31
17:52:48
17:55:05
17:57:23
17:59:40
18:02:00
18:04:18

Output of the top command:

top - 17:53:10 up 2 days, 2:24, 2 users, load average: 1.15, 0.85, 0.96
Tasks: 121 total, 1 running, 120 sleeping, 0 stopped, 0 zombie
Cpu(s): 97.7%us, 2.3%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 1588352k total, 1437304k used, 151048k free, 11480k buffers
Swap: 511992k total, 14096k used, 497896k free, 283060k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6584 zimbra 20 0 983m 173m 12m S 99.1 11.2 0:23.90 java
28092 zimbra 20 0 1485m 385m 14m S 1.0 24.8 0:49.14 java
06-18-2012, 11:59 PM
noondaysun

try running top and then pressing the c key. This shows me more details about the running process. Might help narrow down what is causing the load spike

Standard top

Code:

top - 08:22:21 up 28 min, 2 users, load average: 1.10, 1.02, 0.88 Tasks: 198 total, 1 running, 196 sleeping, 0 stopped, 1 zombie top - 08:57:07 up 1 day, 1:03, 2 users, load average: 0.74, 0.60, 0.75 Tasks: 241 total, 1 running, 239 sleeping, 0 stopped, 1 zombie Cpu(s): 3.2%us, 3.4%sy, 0.0%ni, 58.6%id, 34.7%wa, 0.0%hi, 0.2%si, 0.0%st Mem: 4040108k total, 3892220k used, 147888k free, 35228k buffers Swap: 6384636k total, 435820k used, 5948816k free, 545000k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 6211 root 20 0 87796 67m 1188 D 6 1.7 0:00.45 dpkg 27 root 20 0 0 0 0 S 1 0.0 0:04.77 kswapd0 1297 manline 20 0 1200m 314m 25m S 1 8.0 3:26.76 firefox 3319 manline 20 0 249m 54m 3584 S 1 1.4 0:32.69 skype 1286 manline 20 0 638m 119m 20m S 1 3.0 0:45.50 chromium-browse 2423 root 20 0 197m 19m 9232 S 1 0.5 336:17.02 Xorg 2113 manline 20 0 878m 68m 19m S 0 1.7 0:12.00 chromium-browse

top -c

Code:

manline@feighen:~$ top -c top - 08:58:19 up 1 day, 1:04, 2 users, load average: 0.67, 0.64, 0.76 Tasks: 230 total, 1 running, 228 sleeping, 0 stopped, 1 zombie Cpu(s): 1.2%us, 0.3%sy, 0.0%ni, 98.2%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 4040108k total, 3869864k used, 170244k free, 43464k buffers Swap: 6384636k total, 437056k used, 5947580k free, 572308k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3319 manline 20 0 249m 54m 3416 S 1 1.4 0:33.37 skype 1286 manline 20 0 638m 117m 18m S 1 3.0 0:45.96 /usr/lib/chromium-browser/chromium-browser --type=plugin --plugin-path=/usr/lib/flashplugin-inst 1297 manline 20 0 1200m 314m 24m S 1 8.0 3:28.76 /usr/lib/firefox/firefox 2423 root 20 0 197m 18m 8760 S 1 0.5 336:17.62 /usr/bin/X :0 vt7 -nolisten tcp -auth /var/lib/xdm/authdir/authfiles/A:0-NJe8xU 1209 manline 20 0 855m 46m 18m S 0 1.2 0:02.56 /usr/lib/chromium-browser/chromium-browser --type=renderer --lang=en-GB --force-fieldtest=ConnCo 2354 root 20 0 1201m 164m 2852 S 0 4.2 5:04.70 prosvc 2776 manline 20 0 2162m 519m 14m S 0 13.2 59:53.59 /usr/bin/java -Xms40m -Xmx384m -Dorg.eclipse.equinox.p2.reconciler.dropins.directory=/usr/share/ 3126 manline 20 0 530m 16m 8324 S 0 0.4 2:01.02 lxterminal

Hopefully that helps narrow down which process is causing the spike

Show 40 post(s) from this thread on one page