Java at 100% CPU for two days?

[gkra]

I've noticed that our Zimbra server has been running at an elevated load average for a couple days now, and have tracked it down to two java threads which are both running at 100% CPU (on our 8-core server).

The two threads in particular are:

 $ ps -efL | grep 3577
UID        PID  PPID   LWP  C NLWP STIME TTY          TIME CMD
zimbra   26328 26327  3577 98  257 Jun01 ?        1-03:22:46 /opt/zimbra/java/bin/java -Xms4816m -Xmx4816m -client -XX:NewRatio=2 -Djava.awt.headless=true -XX:MaxPermSize=128m -XX:SoftRefLRUPolicyMSPerMB=1 -Xss256k -Djava.io.tmpdir=/opt/zimbra/mailboxd/work -Djava.library.path=/opt/zimbra/lib -Djava.endorsed.dirs=/opt/zimbra/mailboxd/common/endorsed -Dzimbra.config=/opt/zimbra/conf/localconfig.xml -Djetty.home=/opt/zimbra/mailboxd -DSTART=/opt/zimbra/mailboxd/etc/start.config -jar /opt/zimbra/mailboxd/start.jar /opt/zimbra/mailboxd/etc/jetty.properties /opt/zimbra/mailboxd/etc/jetty-setuid.xml /opt/zimbra/mailboxd/etc/jetty.xml
 $ ps -efL | grep 18198
UID        PID  PPID   LWP  C NLWP STIME TTY          TIME CMD
zimbra   26328 26327 18198 99  250 Jun01 ?        1-03:23:17 /opt/zimbra/java/bin/java -Xms4816m -Xmx4816m -client -XX:NewRatio=2 -Djava.awt.headless=true -XX:MaxPermSize=128m -XX:SoftRefLRUPolicyMSPerMB=1 -Xss256k -Djava.io.tmpdir=/opt/zimbra/mailboxd/work -Djava.library.path=/opt/zimbra/lib -Djava.endorsed.dirs=/opt/zimbra/mailboxd/common/endorsed -Dzimbra.config=/opt/zimbra/conf/localconfig.xml -Djetty.home=/opt/zimbra/mailboxd -DSTART=/opt/zimbra/mailboxd/etc/start.config -jar /opt/zimbra/mailboxd/start.jar /opt/zimbra/mailboxd/etc/jetty.properties /opt/zimbra/mailboxd/etc/jetty-setuid.xml /opt/zimbra/mailboxd/etc/jetty.xml

I can't see anything obvious in the logs about why these two processes are going full-tilt. Is there anything specific I should be looking for?

Is it safe to kill -HUP or -TERM these threads, or should I just let it go until our next maintenance cycle when I can bounce the services?

[veronica]

Do you have Parallel GC enabled ?

zmlocalconfig -e mailboxd_java_options="-client -XX:NewRatio=2 -XX:MaxPermSize=128m -Djava.awt.headless=true -XX:SoftRefLRUPolicyMSPerMB=1 -XX:+UseParallelGC"

for more performance related refer to Performance Tuning Guidelines for Large Deployments - Zimbra :: Wiki

[gkra]

Originally Posted by veronica

Do you have Parallel GC enabled ?

zmlocalconfig -e mailboxd_java_options="-client -XX:NewRatio=2 -XX:MaxPermSize=128m -Djava.awt.headless=true -XX:SoftRefLRUPolicyMSPerMB=1 -XX:+UseParallelGC"

for more performance related refer to Performance Tuning Guidelines for Large Deployments - Zimbra :: Wiki

So, then, this is happening for JVM garbage collection?

Also, I'm assuming that this change won't take effect until I restart the zimbra services, too...

[uxbod]

Yes you will need to restart ZCS.