[SOLVED] Too many open files error

[sdemeyer]

How do I fix this "too many open files" error ? Zimbra dies when this error pops up, which happens nearly every day. I have to restart the server to get Zimbra working again.

Our Zimbra version is:
Release 5.0.2_GA_1975.UBUNTU6 UBUNTU6 NETWORK edition


imap - ignoring error during UID FETCH:
com.zimbra.common.service.ServiceException: system failure: IOException while retrieving content for item 1787
Code:service.FAILURE
at com.zimbra.common.service.ServiceException.FAILURE (ServiceException.java:183)
at com.zimbra.cs.mailbox.MessageCache.getMimeMessage( MessageCache.java:252)
at com.zimbra.cs.mailbox.Message.getMimeMessage(Messa ge.java:290)
at com.zimbra.cs.imap.ImapMessage.getMimeMessage(Imap Message.java:176)
at com.zimbra.cs.imap.ImapHandler.fetch(ImapHandler.j ava:2930)
at com.zimbra.cs.imap.ImapHandler.doFETCH(ImapHandler .java:2781)
at com.zimbra.cs.imap.ImapHandler.executeRequest(Imap Handler.java:402)
at com.zimbra.cs.imap.TcpImapHandler.processCommand(T cpImapHandler.java:151)
at com.zimbra.cs.tcpserver.ProtocolHandler.processCon nection(ProtocolHandler.java:160)
at com.zimbra.cs.tcpserver.ProtocolHandler.run(Protoc olHandler.java:128)
at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Wo rker.run(Unknown Source)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.FileNotFoundException: /opt/zimbra/store/0/13/msg/0/1787-3166.msg (Too many open files)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.jav a:106)
at com.zimbra.cs.store.FileBlobStore.getContent(FileB lobStore.java:366)
at com.zimbra.cs.store.FileBlobStore.getContent(FileB lobStore.java:360)
at com.zimbra.cs.mailbox.MessageCache.fetchFromStore( MessageCache.java:270)
at com.zimbra.cs.mailbox.MessageCache.getMimeMessage( MessageCache.java:214)


Thanks,
Scott

[vandervyvere]

Try setting the open files limit to unlimited. I had similair problems with JBoss cluster servers. It should help. Just follow the steps below.

This setting will set the maximum limit allowed for open files. On the system.
As sudo or root edit the /etc/security/limits.conf file. At the end of the file just above “# End of File” enter the following values:
* soft nofile 65535
* hard nofile 65535
Save the file. This will set the number of open files to unlimited.

Just need to reset the server after this.

[Bill Brock]

It appears you are having trouble with one message in particular. Is it possible the message is corrupt? I''ve seen other mail servers go bonkers over corrupt headers in message files. I believe it tells you where the message is. Can you delete that one message and see if the problem clears up?

[sdemeyer]

Tried the two suggestions but still have the same problem. Zimbra crashed again this morning.

Any more ideas?

[Bill Brock]

After deleting msg 1787, what do the logs say when it crashed. Is it choking on another message?

[jhansen]

Originally Posted by vandervyvere

Try setting the open files limit to unlimited. I had similair problems with JBoss cluster servers. It should help. Just follow the steps below.

This setting will set the maximum limit allowed for open files. On the system.
As sudo or root edit the /etc/security/limits.conf file. At the end of the file just above “# End of File” enter the following values:
* soft nofile 65535
* hard nofile 65535
Save the file. This will set the number of open files to unlimited.

Just need to reset the server after this.

Ubuntu does not include pam_limits.so in common-session, so the open file overrides you have specified in limits.conf are not applied.

To check su to the zimbra user and type 'ulimit -n' if it returns 1024 instead of what you have configured in /etc/security/limits.conf add the following to /etc/pam.d/common-session:

session required pam_limits.so

Restart Zimbra and you should be fine.

[schose]

Exact same problem on debian 4.0.

problem occured with imapsync when migrating mailboxes from old server. Imapsync told me warnings with "no header used or found". when syncing an second time i get varios emails duplicated.

... raise openfiles to 65536 an everything works fine!

[xtrac]

Originally Posted by jhansen

Ubuntu does not include pam_limits.so in common-session, so the open file overrides you have specified in limits.conf are not applied.

To check su to the zimbra user and type 'ulimit -n' if it returns 1024 instead of what you have configured in /etc/security/limits.conf add the following to /etc/pam.d/common-session:

session required pam_limits.so

Restart Zimbra and you should be fine.

jhansen: Thanks for that, Debian 4.0 has the same issue. your fix has been implemented, if it dont work, i'll be back, ulimiy -n now returns the correct number of open files from the limits.conf file.

Thanks muchly

-Roy