Ubuntu 10.04 64bit and ZCS 6.0.9 GA - World of pain

[aseco]

Hi all,

My new installed zimbra server is terrible unstable, the versions are the following:

- Ubuntu 10.04 LTS
- Zimbra NE - ZCS 6.0.9 GA 64bits

I already applied the workarounds found in bugzilla - https://bugzilla.zimbra.com/show_bug.cgi?id=42870, but the problem persists, every 4-5 hours I have to restart the mailbox service. This is completely unacceptable in a production server.

At the moment I don't have the luxury to move zimbra to server (another OS), so I'm stucked with this combination. My clients are very, very unhappy...

Any new ideas to resolve the problem?

[phoenix]

You haven't actually given any details of what the problem is. What about number of user? How much RAM? What's the load on this server? Do you see anything in the log files that might indicate a problem? What does 'top' show, anything interesting? Any other information you can provide that might indicate the area of your problem?

[aseco]

Well, the problem (at least one) is exactly what bug 42870 shows - the bad file descriptor error.

The server is a HP server G5 - quad core with 5G of ram. Only about 40 users on the system - the load on the server is minimum. The server runs without problems for about 4-5 hours, depending on the load, and then the mailboxd service stops responding - only a zmmailboxdctl restart solves the problem.

I have another zimbra installation in another company, is running flawless for more than an year with much more users and in an inferior hardware.

The error on the logs is a java error output:
javax.naming.CommunicationException: simple bind failed: IP address of the server:389 [Root exception is java.net.SocketException: Bad file descriptor]

I also have the following warning, that I find strange:
SL renegotiate denied: java.nio.channels.SocketChannel[connected local=/IP address of the server:7071 remote=/IP address of the server:39589]

I already tested this without any kind of firewall, The server as a public IP address (no NAT), and the external LDAP server is working without flawless - But for the sake of testing I have allready tested with another LDAP server and the problem persists.

Like I told in the last post - changing to another server with another OS is not an option right now, and I can not continue to monitor de server every 5 minutes to see if any problem as occurred - Any advices ?

[Podilarius]

Have you tried to reinstall Zimbra?

Podilarius

[aseco]

This is a fresh install.
No problems during the installation, and it was properly tested before went into production - I didn´t have the change to perform load tests and this problem only appears after a while under some load. If I have the change to reinstall belive me - I will do it (another OS off course), but now that the server is in production and I don't have spare hardware I can't. This clients are very picky about email downtime.

[maxxer]

I ran into this problem myself but with the workaound in the bug solved for good.
I'm running several 609, NE and OSS, on ubuntu 10.04 without any single issue (well, at least not related to ldap).
so double check your settings, must be something related to that!

[danielfarrelly]

Originally Posted by phoenix

What does 'top' show, anything interesting?

What interesting things should we be looking for in 'top'

[phoenix]

Originally Posted by danielfarrelly

What interesting things should we be looking for in 'top'

Anything that would indicate a problem, iowait, cpu usage, RAM usage etc.

[aseco]

Well, the problem persists. If there is no uploads (mails) to the server than it runs fine for more than 24h. If I try to upload more than 1.000 mails it stops responding with the bad file descriptor error.
I noticed that sometimes the mailboxd service restarts itself automatically, but other times it just seats there throwing a lot of java errors.
I really need some help here.

[firemike]

Hi aseco,

as maxxer told you above, you must use the workaround described in Bug 42870(#81):

At OS level:
On Ubuntu 10.0.4 we had added these lines to /etc/security/limits.conf :

root soft nofile 524288
root hard nofile 524288

And we add the command "ulimit -n 524288" in root user .bashrc.