Here's some zmstat charts for a day where java was not being nice,
users were complaining and so
I did a zmmailboxdctl restart at around 1:12pm. After that the rest of
the day it was okay. It's after a few days it seems to get stupid, though
doing an early morning preemptive zmmailboxdctl restart seems to have
good effect.
Summary.txt shows no values, so I don't know what's up there.
(Maybe ubuntu thing)?
===========================================
# Perf Stats #
===========================================
Mail Server CPU Sys :
Mail Server CPU User :
Mail Server CPU Idle :
Mail Server CPU Iowait :
Full GC% :
Young GC% :
Mailbox add rate :
Mailbox add latency :
Mailbox get latency :
Soap response time:
Pop response time:
Imap response time:
but the graphs are nice enough. (I'm modified for hostname = zimbra)
See attached.
Here's a bit of zmmailboxd.out at what I think is the time of the restart.
659891.299: [Full GC 659891.299: [CMS: 1851391K->1851391K(1851392K), 10.6088180 secs] 2406207K->2405946K(2406208K), [CMS Perm : 47486K->47482K(131072K)], 10.6090670 secs] [Times: user=10.32 sys=0.21, real=10.61 secs]
Total time for which application threads were stopped: 10.6108320 seconds
659901.910: [GC [1 CMS-initial-mark: 1851391K(1851392K)] 2405985K(2406208K), 1.0150460 secs] [Times: user=0.93 sys=0.01, real=1.02 secs]
Total time for which application threads were stopped: 1.0162430 seconds
659902.925: [CMS-concurrent-mark-start]
659902.987: [Full GC 659902.987: [CMS659914.927: [CMS-concurrent-mark: 11.999/12.001 secs] [Times: user=8.06 sys=0.03, real=12.00 secs]
(concurrent mode failure): 1851391K->1851391K(1851392K), 24.2305050 secs] 2406207K->2405670K(2406208K), [CMS Perm : 47482K->47480K(131072K)], 24.2308000 secs] [Times: user=18.57 sys=0.30, real=24.23 secs]
Total time for which application threads were stopped: 24.2324320 seconds
Heap
par new generation total 554816K, used 554279K [0x15660000, 0x3b060000, 0x3b060000)
eden space 493184K, 99% used [0x15660000, 0x337ffff0, 0x33800000)
from space 61632K, 99% used [0x33800000, 0x373a9c50, 0x37430000)
to space 61632K, 0% used [0x37430000, 0x37430000, 0x3b060000)
concurrent mark-sweep generation total 1851392K, used 1851391K [0x3b060000, 0xac060000, 0xac060000)
concurrent-mark-sweep perm gen total 131072K, used 47480K [0xac060000, 0xb4060000, 0xb4060000)
659927.238: [Full GC 659927.238: [CMS: 1851391K->1851391K(1851392K), 11.5150760 secs] 2406202K->2405705K(2406208K), [CMS Perm : 47483K->47483K(13107\
2K)], 11.5157270 secs] [Times: user=10.32 sys=0.30, real=11.52 secs]
CompilerOracle: exclude com/zimbra/cs/session/SessionMap.putAndPrune
CompilerOracle: exclude com/zimbra/cs/mailbox/MailItem.delete
CompilerOracle: exclude org/apache/xerces/impl/XMLDocumentFragmentScannerImpl$FragmentContentDisp atcher.dispatch
Total time for which application threads were stopped: 0.0004250 seconds
0 INFO [main] log - Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
Total time for which application threads were stopped: 0.0003980 seconds
Total time for which application threads were stopped: 0.0004740 seconds
Zimbra server reserving server socket port=110 bindaddr=null ssl=false
Zimbra server reserving server socket port=995 bindaddr=null ssl=true
Zimbra server reserving server socket port=143 bindaddr=null ssl=false
Zimbra server reserving server socket port=993 bindaddr=null ssl=true
Zimbra server reserving server socket port=7025 bindaddr=null ssl=false
Total time for which application threads were stopped: 0.0009940 seconds
5242 INFO [main] log - Setting umask=027
5244 INFO [main] log - Current rlimit_nofiles (soft=524288, hard=524288)
5244 INFO [main] log - Set rlimit_nofiles (soft=65536, hard=65536)
5244 INFO [main] log - Setting GID=1005
before that I was getting things like
CMS: abort preclean due to time 659306.572: [CMS-concurrent-abortable-preclean: 2.999/5.137 secs] [Times: user=2.26 sys=0.08, real=5.14 secs]
659306.573: [GC[YG occupancy: 541888 K (554816 K)]659306.573: [Rescan (parallel) , 2.0809960 secs]659308.655: [weak refs processing, 0.0000260 secs] [1 CMS-remark: 1851379K(1851392K)] 2393268K(2406208K), 2.0814040 secs] [Times: user=1.20 sys=0.72, real=2.08 secs]
Total time for which application threads were stopped: 2.0826130 seconds
and
659136.042: [GC 659136.042: [ParNew: 554815K->554815K(554816K), 0.0000730 secs]659136.042: [CMS659141.193: [CMS-concurrent-mark: 7.484/7.487 secs] [Times: user=7.78 sys=0.19, real=7.48 secs]
(concurrent mode failure): 1851378K->1851391K(1851392K), 16.7058680 secs] 2406193K->2375479K(2406208K), [CMS Perm : 47477K->47474K(131072K)], 16.7062600 secs] [Times: user=16.38 sys=0.24, real=16.71 secs]
Total time for which application threads were stopped: 16.7076830 seconds
thanks,
John |