Pop dies randomly on suse EL 9

[kechols]

I'm running suse 9 EL with zimbra 3.1.0 (I know it's an old version) in order to get the ok to upgrade I need to be able to show my boss it will help with the problem we have.

For some reason we have POP die once every week or two at random times. I can't find any reason for this, for a while I thought it was a network issue (large numbers of clients not closing the connection properly). Stopping zimbra, syncing filesystems and restarting zimbra was enough to clear it up.

Lately it's gotten worse, the last time this happened was between 2:00 and 2:08 pm today and I can't find anything in the logs showing what's causing it. Due to the apparently random nature I can't readily pull a stack trace.

The server is stopped everynight about 4am for backup purposes and restarted as soon as backups are complete so I don't think it's an out of memory problem, otherwise it should happen more frequently (or am i mistaken here?).

I'm attaching output from /var/log/zimbra.log , cpuinfo, and meminfo

If anyone can point me in the direction of either a fix or help me figure out at least what is causing it PLEASE do. I've been fighting with this for about 2 1/2 months with no luck. Even searching forums I've found similar problems but they were memory related and I see no java out of memory errors in logs.

Thanks,
Kyle

[dkarp]

What's zimbraPop3NumThreads set to in LDAP? Does raising that number solve your problem?

What does kill -3 say about the currently-active POP threads?

[kechols]

zmprov gacf |grep zimbraPop3NumThreads -i
zimbraPop3NumThreads: 20
zimbraServerInheritedAttr: zimbraPop3NumThreads

How high should I be able safely raise it without problems?

[kechols]

As I understand it, zimbraPop3NumThreads is the number of threads allowed to run for pop3 services. and each thread can handle 10 connections at a time assuming I'm correct on this, I can have 200 connections at a time and with my userbase (1800 accounts active right now) I probably need to up it to at least 30 - 35 maybe 40 threads. Before I turn my thread count up I want to make sure my math is correct. If I'm not right on this please let me know and I'll recalculate.

Thanks,
kyle

[dkarp]

The 3.1.4 POP3 server is thread-per-connection. This is usually fine, as POP3 connections aren't long-lived. But if your POP clients aren't disconnecting properly, you'll need to increase this number a fair amount (and fix the clients!). Try upping the thread count to 80 and see if the problem goes away...

[kechols]

I"ve upped it to 40 so i can see what memory utilization will be. After a couple of days with no problems I'll up it again to 60 then to 80 a few days later. So far setting it to 40 seems to have helped recovery times significantly when pop was dieing. Thanks for the clarification.

Kyle