OutOfMemory

[i2ambler]

2011-07-27 12:24:20,775 INFO [main] [] index - Initialized Index for mailbox 2061 directory: LuceneIndex at com.zimbra.cs.index.Z23FSDirectory@/opt/zimbra/index/0/2061/index/0 Analyzer=com.zimbra.cs.index.ZimbraAnalyzer@370aa2 82
2011-07-27 12:24:20,775 INFO [main] [] cache - initializing folder and tag caches for mailbox 2061
2011-07-27 12:24:20,782 INFO [main] [] mbxmgr - Mailbox 2061 account a2f2cfe8-bc36-4f90-9757-b42934012009 LOADED
2011-07-27 12:24:21,365 FATAL [main] [] system - out of memory
java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.index.TermInfosReader.ensureInde xIsRead(TermInfosReader.java:173)
at org.apache.lucene.index.TermInfosReader.get(TermIn fosReader.java:225)
at org.apache.lucene.index.TermInfosReader.get(TermIn fosReader.java:218)
at org.apache.lucene.index.SegmentTermDocs.seek(Segme ntTermDocs.java:55)
at org.apache.lucene.index.IndexReader.termDocs(Index Reader.java:780)
at org.apache.lucene.index.DocumentsWriter.applyDelet es(DocumentsWriter.java:952)
at org.apache.lucene.index.DocumentsWriter.applyDelet es(DocumentsWriter.java:918)
at org.apache.lucene.index.IndexWriter.applyDeletes(I ndexWriter.java:4339)
at org.apache.lucene.index.IndexWriter.doFlush(IndexW riter.java:3579)
at org.apache.lucene.index.IndexWriter.flush(IndexWri ter.java:3450)


This is causing the mailboxd to hang - and a restart is required. I have 16 gig of ram and am set to heap memory percent = 25.. Which is what I understand to be sufficient??

[soba@ukw.edu.pl]

What is your OS and OS version? You use virtual machine?
if you are using a virtual machine. If so, what solutions?

[i2ambler]

Originally Posted by soba@ukw.edu.pl

What is your OS and OS version? You use virtual machine?
if you are using a virtual machine. If so, what solutions?

I am using redhat 5 - virtual machine on vsphere 4.0. 2vcpu, 16gb ram. I have not had any solution or suggestion about this problem.

To be honest, 70% of my posts here go unanswered.

[phoenix]

Originally Posted by i2ambler

I am using redhat 5 - virtual machine on vsphere 4.0. 2vcpu, 16gb ram. I have not had any solution or suggestion about this problem.

Have you considered raising a support case for this? Can you confirm the jave allocations with the output of this:

zmlocalconfig | grep -i mailboxd_java

Does this happen at any specific time of day? With any specific mailbox? Is there anything about this that you can reproduce? What's the state of this VM at the time of the problem (have you looked at top)? Any RAID system in this VM? Have you looked at any of the performance metrics in vSphere for this esx server and the VM?Have you considered upgrading to the most recent release of Zimbra? Have you looked at this page for Performance Recommendations. Have you had this problem for a while or has it just started? Any specific upgrades to the operating system on the zimbra server?

Originally Posted by i2ambler

To be honest, 70% of my posts here go unanswered.

Perhaps the reason might be that nobody here knows the answer?

[i2ambler]

Originally Posted by phoenix

Have you considered raising a support case for this? Can you confirm the jave allocations with the output of this:

zmlocalconfig | grep -i mailboxd_java

Does this happen at any specific time of day? With any specific mailbox? Is there anything about this that you can reproduce? What's the state of this VM at the time of the problem (have you looked at top)? Any RAID system in this VM? Have you looked at any of the performance metrics in vSphere for this esx server and the VM?Have you considered upgrading to the most recent release of Zimbra? Have you looked at this page for Performance Recommendations. Have you had this problem for a while or has it just started? Any specific upgrades to the operating system on the zimbra server?


Perhaps the reason might be that nobody here knows the answer?

I have a support ticket about this.. I have used all performance recommendations recommended to me by zimbra for both VM and for the OS itself. Support has examined our environment and assured us that the environment should be well able to handle the current user load. (we have 4000 users spread across 2 mailstores that have 16 gigs ram, 10 spindles of 15k fiber drives per TB of data, and 2 dedicated cores per server. The system basically flies 90 percent of the time. All crashes are random, all slowdowns appear to be random and not at any specified time. Last week we had a few minutes where the load went up to 19 on one system, concurrent with 700 soap calls/min. This was occuring for about 5 minutes during the day. We normally average 50-75 soap calls/min.

Currently java is using 52% of RAM on one of my mailstores, even though I have the heap set to 25 percent, and am running the most recent version of 6, being 6.0.13. (but have not updated my profile here to reflect that yet).

mailboxd_java_heap_memory_percent = 25
mailboxd_java_heap_new_size_percent = 25


All issues began when we finally fully rolled out our users. We had no issue when only running about 1000 users.

I understand the forum is very helpful, and has helped me in the past, but some of my issues have been very frustrating - and this performance issue is a huge thorn in my a$$.

[iway]

Have you found a solution to this yet?