Sudden performance problems

[ryanmr-gc]

I'm running up Zimbra (5.0.15) and have suddenly started experiencing major performance problems. Starting yesterday lots of operations started lagging. Any time you move an email to another folder (Imap and web client), send a message (web client), or create/delete a calendar appointment, the processes hangs for 5-30 seconds before completing. The box itself is not loaded and as far as I can tell is not suffering from lagging drives, bad networking, etc. Everything else on the box seems to be fine and it appears to be an internal zimbra problem. There are only 15 users on the box right now and until yesterday everything was working well. Nothing has been changed in Zimbra or the OS. Has anyone else seen this or have any ideas as to how to figure it out? I can find no indication in any logs that anything is amiss.

Release 5.0.15_GA_2851.UBUNTU8_64 UBUNTU8_64 NETWORK edition
running on Ubuntu 8.04
2 X Quad-Core AMD Opteron 2376
16G RAM

[iway]

Have a look at this thread: Zimbra on Ubuntu 8.0.4 - severe issues
Might be the same...
You could vote for the bug as well!

[ryanmr-gc]

Originally Posted by interways

Have a look at this thread: Zimbra on Ubuntu 8.0.4 - severe issues
Might be the same...
You could vote for the bug as well!

Looks like a different issue to me. According to TOP i've still got 11G of ram free and restarting doesn't seem to affect it.

[tgx]

Originally Posted by ryanmr-gc

Looks like a different issue to me. According to TOP i've still got 11G of ram free and restarting doesn't seem to affect it.

Firefox or IE web clients? Just wondering if Firefox security update is impacting you?

[ryanmr-gc]

Originally Posted by tgx

Firefox or IE web clients? Just wondering if Firefox security update is impacting you?

Once it starts getting slow it drags no matter which browser I've tried. Firefox 3.0/3.1 (win and linux), safari (mac), or chrome (win).

[uxbod]

Okay so how about some system information.

top
cat /proc/meminfo
freemem

su - zimbra
zmlocalconfig mailboxd_java_heap_memory_percent

[ryanmr-gc]

Top:
top - 10:57:35 up 1 day, 14:18, 1 user, load average: 0.71, 0.62, 0.44
Tasks: 201 total, 1 running, 200 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.5%us, 0.1%sy, 0.0%ni, 99.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 16332204k total, 6217452k used, 10114752k free, 344456k buffers
Swap: 33559776k total, 0k used, 33559776k free, 1829816k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
29809 zimbra 20 0 7030m 2.6g 46m S 2 17.0 8:57.72 java
28487 zimbra 20 0 4269m 633m 4812 S 1 4.0 4:21.72 mysqld
3009 root 15 -5 0 0 0 S 0 0.0 0:04.77 kjournald
6835 root 20 0 19060 1372 960 R 0 0.0 0:00.11 top
31272 zimbra 20 0 35036 7484 1176 S 0 0.0 0:13.37 zmmtaconfig
1 root 20 0 4088 888 604 S 0 0.0 0:01.91 init
2 root 15 -5 0 0 0 S 0 0.0 0:00.00 kthreadd
3 root RT -5 0 0 0 S 0 0.0 0:00.37 migration/0
4 root 15 -5 0 0 0 S 0 0.0 0:00.03 ksoftirqd/0
5 root RT -5 0 0 0 S 0 0.0 0:00.00 watchdog/0
6 root RT -5 0 0 0 S 0 0.0 0:00.49 migration/1
7 root 15 -5 0 0 0 S 0 0.0 0:00.05 ksoftirqd/1
8 root RT -5 0 0 0 S 0 0.0 0:00.00 watchdog/1
9 root RT -5 0 0 0 S 0 0.0 0:00.43 migration/2
10 root 15 -5 0 0 0 S 0 0.0 0:00.03 ksoftirqd/2
11 root RT -5 0 0 0 S 0 0.0 0:00.00 watchdog/2
12 root RT -5 0 0 0 S 0 0.0 0:00.50 migration/3
13 root 15 -5 0 0 0 S 0 0.0 0:00.02 ksoftirqd/3
14 root RT -5 0 0 0 S 0 0.0 0:00.00 watchdog/3
15 root RT -5 0 0 0 S 0 0.0 0:00.56 migration/4
16 root 15 -5 0 0 0 S 0 0.0 0:00.03 ksoftirqd/4
17 root RT -5 0 0 0 S 0 0.0 0:00.00 watchdog/4
18 root RT -5 0 0 0 S 0 0.0 0:00.46 migration/5
19 root 15 -5 0 0 0 S 0 0.0 0:00.07 ksoftirqd/5
20 root RT -5 0 0 0 S 0 0.0 0:00.00 watchdog/5
21 root RT -5 0 0 0 S 0 0.0 0:00.49 migration/6
22 root 15 -5 0 0 0 S 0 0.0 0:00.01 ksoftirqd/6
23 root RT -5 0 0 0 S 0 0.0 0:00.00 watchdog/6


cat /proc/meminfo
MemTotal: 16332204 kB
MemFree: 10113992 kB
Buffers: 344508 kB
Cached: 1830140 kB
SwapCached: 0 kB
Active: 4575228 kB
Inactive: 1145924 kB
Active(anon): 3546764 kB
Inactive(anon): 0 kB
Active(file): 1028464 kB
Inactive(file): 1145924 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 33559776 kB
SwapFree: 33559776 kB
Dirty: 468 kB
Writeback: 0 kB
AnonPages: 3546464 kB
Mapped: 72292 kB
Slab: 427776 kB
SReclaimable: 384732 kB
SUnreclaim: 43044 kB
PageTables: 12676 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 41725876 kB
Committed_AS: 10461888 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 295412 kB
VmallocChunk: 34359441903 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 10176 kB
DirectMap2M: 16898048 kB

mailboxd_java_heap_memory_percent = 40

Of course, right this second it happens to be behaving. I'll post again when it starts slowing down again.