IMAP server stop processing

[slaplante]

In the last 2 weeks, I have the imap server process that keep answering to TCP 143, as I am able to connect to it with telnet, but then after the connection; nothing. Like the process go zombie.

I check the /var/log/maillog but aside the last transactions that been log there no error.

However, everything else looks to work fine during the imap hang WebUI, mailbox, LDAP...

Restarting Zimbra resumed the issue but it happened for the third time in 2 weeks, so I'm afraid of a degradation...

I use Zimbra 6.0.6_GA_2330 on a RHEL 4 (Nahant Update 5)

Any hint where I can look to identify what went wrong.

Thanks!

Steeve

[iway]

Have you had a look at imapmaxnumthreads?

[slaplante]

Here's the setting;
zimbraImapNumThreads: 200

Is it too low? On big day, we have about 125 concurrent connections. What the implication to up it on memory ie?

Thank very much for your help!

Steeve

[iway]

Well, we have it set at 500 currently.

[slaplante]

Hi,

Sorry for the delay of answer but I had to let run the suggested setting to see the result. Unfortunately, after two busy Mondays, it doesn't seem the increase to 500 concurrents connections helps.

Any other suggestion or log I can look into to help?

Thanks greatly guys!

Steeve

[dalmate]

When the imap service dies, did you check how many connections are connecting to your server:
netstat -ant | grep 143 | wc -l
as zimbra user.
Please let's me know that information and I can help.

[slaplante]

Dalmate,

Connections ran as high as 300 in the peak hour. Server scaled well to 175 but start to crawl and became slow in the 200 connections.

I also saw that the swap is almost not use but there was memory left so I guess it is normal.

It worth notice that Zimbra main process went often over 100% CPU utilization?!?!

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22672 zimbra 16 0 2964m 2.6g 11m S 150 45.6 1314:59 java

Wierd...

Steeve

[y@w]

Dalmate,

Connections ran as high as 300 in the peak hour. Server scaled well to 175 but start to crawl and became slow in the 200 connections.

I also saw that the swap is almost not use but there was memory left so I guess it is normal.

It worth notice that Zimbra main process went often over 100% CPU utilization?!?!

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22672 zimbra 16 0 2964m 2.6g 11m S 150 45.6 1314:59 java

Wierd...

Steeve

The workload is being spread across multiple cores/processors

What's a little more interesting is that java is using 45% of your memory. It shouldn't need to be quite that high (and can cause problems if it is). How much memory is in this system? See: Performance Tuning Guidelines for Large Deployments - Zimbra :: Wiki

Also, anything interesting in /opt/zimbra/log/mailbox.log when the problem occurs?

[dalmate]

@slaplante:
How many Imap concurrent connections did you set?
Please check by this command:
zmprov gs yourdomain.com | grep Threads
if it is 200 threads, please increase to 500 threads (or any number you want).
About your CPU, please let me know which version of zimbra did you use?

[slaplante]

Thank y@w, dalmate,

We are at about 6Gb of RAM and I'll upgrade it to 8Gb. I also have all settings referenced in the link provided and have the server rebooted tonight. I will see the outcome next monday.

dalmate, As suggested I upped the imap threads from 200 to 500 with no concrete improvement.

Thanks again guys, I'll keep you post with the results next monday...

Steeve