Page 1 of 2 12 LastLast
Results 1 to 10 of 15

Thread: System out of memory after tnef mapping

  1. #1
    Twisteeer is offline Member
    Join Date
    Aug 2013
    Posts
    11
    Rep Power
    1

    Default System out of memory after tnef mapping

    Guys,
    I've posted on a closed thread and now I'm opening a new one in order to do the right thing. [Newbie]

    Version info :
    Release 7.1.4_GA_2555.RHEL6_64_20120105094542 CentOSUNKNOWN_64 FOSS edition.
    Running under VmWare ESX 4.1.

    Problem :
    There's a specific account which has a problem as the log (/opt/zimbra/log/mailbox.log) says below. With a message being recognized as a TNEF message possibly an ICalendar message being unprocessed. But this is putting the processing at the top and makes the server hang for a while.
    I've already tried to change the javamail_zparser on zmmailboxdctl [that was true I've put false], with no success. I'm trying to figure out something else to do, and by now I have nothing on my mind. I need some help !
    Thanks in advance


    2013-08-12 08:10:36,574 INFO [LmtpServer-3] [name=account@domain.com;mid=87;ip=192.168.0.x;] tnef - Calendaring TNEF message mapped to ICALENDAR with UID=040000008200E00074C5B7101A82E00800000000809AA2 AF68D8CD01000000000000000010000000BB62690EDF0A8F40 A2F9BFB776EB68A3
    2013-08-09 08:10:36,574 INFO [LmtpServer-3] [name=account@domain.com;mid=87;ip=192.168.0.x;] tnef - Calendaring TNEF message mapped to ICALENDAR with UID=040000008200E00074C5B7101A82E00800000000809AA2 AF68D8CD01000000000000000010000000BB62690EDF0A8F40 A2F9BFB776EB68A3
    2013-08-09 08:10:38,648 INFO [LmtpServer-3] [name=account@domain;mid=87;ip=192.168.0.x;] tnef - Calendaring TNEF message mapped to
    ICALENDAR with UID=040000008200E00074C5B7101A82E00800000000809AA2 AF68D8CD01000000000000000010000000BB62690EDF0A8F40 A2F9BFB776EB68A3


    2013-08-12 08:14:41,323 WARN [LmtpServer-20] [name=account@domain.com;mid=87;ip=192.168.0.x;] ParsedMessage - Unable to parse

    part=2.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.2 1 filename= content-type=message/rfc822 message-

    id=<486340632.89858.1376306076250.JavaMail.root@se rver.domain.com>
    com.zimbra.cs.mime.MimeHandlerException: No content
    at com.zimbra.cs.mime.handler.MessageRFC822Handler.ge tContentImpl(MessageRFC822Handler.java:73)
    at com.zimbra.cs.mime.MimeHandler.getContent(MimeHand ler.java:157)
    at com.zimbra.cs.mime.ParsedMessage.analyzePart(Parse dMessage.java:1060)
    at com.zimbra.cs.mime.ParsedMessage.analyzeNonBodyPar ts(ParsedMessage.java:420)
    at com.zimbra.cs.mime.ParsedMessage.analyzeFully(Pars edMessage.java:454)
    at com.zimbra.cs.mailbox.CalendarItem.getIndexDocumen ts(CalendarItem.java:380)
    at com.zimbra.cs.mailbox.CalendarItem.generateIndexDa ta(CalendarItem.java:258)
    at com.zimbra.cs.mailbox.IndexHelper.indexItemList(In dexHelper.java:828)
    at com.zimbra.cs.mailbox.IndexHelper.indexDeferredIte msInternal(IndexHelper.java:354)
    at com.zimbra.cs.mailbox.IndexHelper.indexDeferredIte ms(IndexHelper.java:274)
    at com.zimbra.cs.mailbox.IndexHelper.maybeIndexDeferr edItems(IndexHelper.java:230)
    at com.zimbra.cs.mailbox.Mailbox.addMessage(Mailbox.j ava:4697)
    at com.zimbra.cs.mailbox.Mailbox.addMessage(Mailbox.j ava:4690)
    at com.zimbra.cs.mailbox.Mailbox.addMessage(Mailbox.j ava:4638)
    at com.zimbra.cs.filter.IncomingMessageHandler.addMes sage(IncomingMessageHandler.java:122)
    at com.zimbra.cs.filter.IncomingMessageHandler.implic itKeep(IncomingMessageHandler.java:116)
    at com.zimbra.cs.filter.ZimbraMailAdapter.doDefaultFi ling(ZimbraMailAdapter.java:343)
    at com.zimbra.cs.filter.ZimbraMailAdapter.executeActi ons(ZimbraMailAdapter.java:215)
    at org.apache.jsieve.SieveFactory.evaluate(SieveFacto ry.java:173)
    at com.zimbra.cs.filter.RuleManager.applyRulesToIncom ingMessage(RuleManager.java:401)
    at com.zimbra.cs.filter.RuleManager.applyRulesToIncom ingMessage(RuleManager.java:363)
    at com.zimbra.cs.lmtpserver.ZimbraLmtpBackend.deliver MessageToLocalMailboxes(ZimbraLmtpBackend.java:571 )
    at com.zimbra.cs.lmtpserver.ZimbraLmtpBackend.deliver (ZimbraLmtpBackend.java:370)
    at com.zimbra.cs.lmtpserver.LmtpHandler.processMessag eData(LmtpHandler.java:367)
    at com.zimbra.cs.lmtpserver.TcpLmtpHandler.continueDA TA(TcpLmtpHandler.java:72)
    at com.zimbra.cs.lmtpserver.LmtpHandler.doDATA(LmtpHa ndler.java:356)
    at com.zimbra.cs.lmtpserver.LmtpHandler.processComman d(LmtpHandler.java:172)
    at com.zimbra.cs.lmtpserver.TcpLmtpHandler.processCom mand(TcpLmtpHandler.java:65)
    at com.zimbra.cs.tcpserver.ProtocolHandler.processCon nection(ProtocolHandler.java:199)
    at com.zimbra.cs.tcpserver.ProtocolHandler.run(Protoc olHandler.java:140)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run Task(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.io.IOException: No content
    at javax.mail.internet.MimePartDataSource.getInputStr eam(MimePartDataSource.java:108)
    at com.zimbra.cs.mime.handler.MessageRFC822Handler.ge tContentImpl(MessageRFC822Handler.java:57)
    ... 32 more


    2013-08-12 08:18:11,940 FATAL [LmtpServer-20] [name=account@domain.com;mid=87;ip=192.168.0.x;] system - out of memory
    java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2882)
    at java.lang.AbstractStringBuilder.expandCapacity(Abs tractStringBuilder.java:100)
    at java.lang.AbstractStringBuilder.append(AbstractStr ingBuilder.java:390)
    at java.lang.StringBuilder.append(StringBuilder.java: 119)
    at com.zimbra.cs.mime.ParsedMessage.appendToContent(P arsedMessage.java:1141)
    at com.zimbra.cs.mime.ParsedMessage.analyzeNonBodyPar ts(ParsedMessage.java:422)
    at com.zimbra.cs.mime.ParsedMessage.analyzeFully(Pars edMessage.java:454)
    at com.zimbra.cs.mailbox.CalendarItem.getIndexDocumen ts(CalendarItem.java:380)
    at com.zimbra.cs.mailbox.CalendarItem.generateIndexDa ta(CalendarItem.java:258)
    at com.zimbra.cs.mailbox.IndexHelper.indexItemList(In dexHelper.java:828)
    at com.zimbra.cs.mailbox.IndexHelper.indexDeferredIte msInternal(IndexHelper.java:354)
    at com.zimbra.cs.mailbox.IndexHelper.indexDeferredIte ms(IndexHelper.java:274)
    at com.zimbra.cs.mailbox.IndexHelper.maybeIndexDeferr edItems(IndexHelper.java:230)
    at com.zimbra.cs.mailbox.Mailbox.addMessage(Mailbox.j ava:4697)
    at com.zimbra.cs.mailbox.Mailbox.addMessage(Mailbox.j ava:4690)
    at com.zimbra.cs.mailbox.Mailbox.addMessage(Mailbox.j ava:4638)
    at com.zimbra.cs.filter.IncomingMessageHandler.addMes sage(IncomingMessageHandler.java:122)
    at com.zimbra.cs.filter.IncomingMessageHandler.implic itKeep(IncomingMessageHandler.java:116)
    at com.zimbra.cs.filter.ZimbraMailAdapter.doDefaultFi ling(ZimbraMailAdapter.java:343)
    at com.zimbra.cs.filter.ZimbraMailAdapter.executeActi ons(ZimbraMailAdapter.java:215)
    at org.apache.jsieve.SieveFactory.evaluate(SieveFacto ry.java:173)
    at com.zimbra.cs.filter.RuleManager.applyRulesToIncom ingMessage(RuleManager.java:401)
    at com.zimbra.cs.filter.RuleManager.applyRulesToIncom ingMessage(RuleManager.java:363)
    at com.zimbra.cs.lmtpserver.ZimbraLmtpBackend.deliver MessageToLocalMailboxes(ZimbraLmtpBackend.java:571 )
    at com.zimbra.cs.lmtpserver.ZimbraLmtpBackend.deliver (ZimbraLmtpBackend.java:370)
    at com.zimbra.cs.lmtpserver.LmtpHandler.processMessag eData(LmtpHandler.java:367)
    at com.zimbra.cs.lmtpserver.TcpLmtpHandler.continueDA TA(TcpLmtpHandler.java:72)
    at com.zimbra.cs.lmtpserver.LmtpHandler.doDATA(LmtpHa ndler.java:356)
    at com.zimbra.cs.lmtpserver.LmtpHandler.processComman d(LmtpHandler.java:172)
    at com.zimbra.cs.lmtpserver.TcpLmtpHandler.processCom mand(TcpLmtpHandler.java:65)
    at com.zimbra.cs.tcpserver.ProtocolHandler.processCon nection(ProtocolHandler.java:199)
    at com.zimbra.cs.tcpserver.ProtocolHandler.run(Protoc olHandler.java:140)

  2. #2
    liverpoolfcfan's Avatar
    liverpoolfcfan is offline Outstanding Member
    Join Date
    Oct 2009
    Location
    Dublin, IRELAND
    Posts
    710
    Rep Power
    6

    Default

    Is it a well formed message ?

    I have never seen a message with so many embedded MIME parts - part=2.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.2

    It seems like it is just running out of memory trying to keep all those layers in memory as it parses. How many amavisd processes do you have running on your system (10 by default) - If you are running a small server/user group you could reduce this down to 2 or 3 to free up memory in the VM to see if it gets enough for java to process the message. The setting is in /opt/zimbra/conf/

    $max_servers = 2; # number of pre-forked children (2..15 is common)

    Set it to a lower value and restart. Each of those processes utilizes a large chunk of memory.

    Next option might be to connect with an IMAP client - and see if it can process the message. Then forward the pieces of the original that are needed, and delete the original.

    If that doesn't work either, your best bet might be command line soap tools to possibly forward the message out of the system if it lets you, and then delete it. Look for soap.txt in /opt/zimbra/docs for details of the soap messages, and the command line too zmsoap for instructions on how to call them.

  3. #3
    Twisteeer is offline Member
    Join Date
    Aug 2013
    Posts
    11
    Rep Power
    1

    Default

    First of all, I would like to thank you for your help.
    I was already lost ! LOL
    So, I don't think it is a well formed message. If there's no other way, I'll have to delete it. I was also trying to figure out a way to do that.
    And yes, I've changed the value $max_servers in amavis.conf and also in amavis.conf.in inside /opt/zimbra/conf to "3". The value was set in 10 (default).
    Is it right ? Because I've noticed this file called amavis.conf.if with the same configuration, and maybe if I restart the server with this amavis.conf.in unchanged, this could re-configurate the first one I've changed (amavis.conf).
    I work in a small/medium company. We have 800 accounts, and then it will take a little while to be able to restart the server.
    How could I connect an IMAP client into the user account to do the second step, if the first one fails ? Like, is it with the normal process, changing the user's password and connecting a Zimbra Desktop / Outlook / Thunderbird ?
    I'm sorry but I'm not experienced with messaging servers.
    I'll let you know when I restart the server and thanks again !

  4. #4
    liverpoolfcfan's Avatar
    liverpoolfcfan is offline Outstanding Member
    Join Date
    Oct 2009
    Location
    Dublin, IRELAND
    Posts
    710
    Rep Power
    6

    Default

    on startup the system reads amavisd.conf.in and uses the settings in there to create the actual runtime configuration file amavisd.conf - changes made directly in amavisd.conf will get overwritten on the next startup before they can ever be actioned.

    With 800 users 3 might be too small. You can watch the mail queues and see if emails are getting backend up in the Incoming queue. But 10 is most likely too big. Somewhere in the middle might be best for you.

    IMAP - yes. Make sure you have configured the server to allow IMAP and use Thunderbird or similar to open the user's mailbox.

  5. #5
    Twisteeer is offline Member
    Join Date
    Aug 2013
    Posts
    11
    Rep Power
    1

    Default

    Yes, thanks for the informations.
    And I did those steps but the server keeps me giving those log messages and the processing goes at the top.
    I've changed the amavis.conf to 3 and restarted the server. If it becomes unmanageable I'll put it up.
    I've connected to the user's mailbox with Thunderbird using an IMAP configuration and the processing went crazy, but it stills in there. That malformed message stills giving me trouble.
    I wanted to kindly ask you, what was the point of connecting with an IMAP Client ? To see if the client computer could help in the processing ?

    I've already tried before to use zmmailbox with "get message (gm)" option or even "delete message" , but I couldn't do this.
    Now I think that we have no other choices.
    How can I delete/forward this message using command line tools ?
    I've tried to search for something on soap.txt as you said before, but I found nothing.

    Following the log traces (mailbox.log) I've noticed some messages like this one :
    013-08-13 12:31:14,209 INFO [btpool0-45://mail/service/soap/SearchRequest] [name=account@domain.com;mid=616;ip=192.168.0.x;ua= ZimbraWebClient - IE8 (Win)/7.1.4_GA_2555;] sqltrace - Slow execution (10693ms): SELECT mi.id, mi.index_id, mi.type, mi.date AS sortcol, mi.id, mi.type, mi.parent_id, mi.folder_id, mi.index_id, mi.imap_id, mi.date, mi.size, mi.volume_id, mi.blob_digest, mi.unread, mi.flags, mi.tags, mi.subject, mi.name, mi.metadata, mi.mod_metadata, mi.change_date, mi.mod_content FROM mboxgroup16.mail_item AS mi FORCE INDEX (i_folder_id_date) WHERE mi.mailbox_id = 616 AND ((type = 16 OR type = 5) AND mi.folder_id = 2) ORDER BY sortcol DESC LIMIT 0,103

    I'll keep trying to do something here.
    See ya.

  6. #6
    Twisteeer is offline Member
    Join Date
    Aug 2013
    Posts
    11
    Rep Power
    1

    Default

    I tried to use zmsoap but I need more info.
    Can someone give me examples of how to use that ?
    I need someone to help me out.
    Regards
    Last edited by Twisteeer; 08-16-2013 at 06:00 AM. Reason: Improving

  7. #7
    Twisteeer is offline Member
    Join Date
    Aug 2013
    Posts
    11
    Rep Power
    1

    Default

    There is someone to help me out there ?

  8. #8
    Twisteeer is offline Member
    Join Date
    Aug 2013
    Posts
    11
    Rep Power
    1

    Default

    I have solved the problem.
    Unfortunately I couldn't solve it technologically, because there was more messages than that one "corrupted."
    People talked and sent several times the corrupted message(s).
    I've asked to the user to delete "calendar messages".
    And after that, the server became stable again.
    Thank you guys.
    I need more training and experience, but your help was essential.

  9. #9
    quanah is offline Zimbra Employee
    Join Date
    May 2007
    Location
    Zimbra
    Posts
    1,271
    Rep Power
    10

    Default

    I strongly advise upgrading to a current release as well.
    Quanah Gibson-Mount
    Server Architect
    Zimbra, Inc
    --------------------
    Zimbra :: the leader in open source messaging and collaboration

  10. #10
    Twisteeer is offline Member
    Join Date
    Aug 2013
    Posts
    11
    Rep Power
    1

    Default

    I'll plan to do so.
    But I have to say that I was mistaken. And now there's some e-mails in the queue to this user and the processing goes high again.
    And everytime I try to do some operation on this account, the server hangs for a while.

    362510 2013-08-21 14:39:39,954 INFO [main] [] tnef - Calendaring TNEF message mapped to ICALENDAR with UID=040000008200E00074C5B7101A82E00800000000809AA2 AF68D8CD01000000000000000010000000BB62690EDF0A8F40 A2F9BFB776EB68A3
    362511 2013-08-21 14:39:41,029 INFO [main] [] tnef - Calendaring TNEF message mapped to ICALENDAR with UID=040000008200E00074C5B7101A82E00800000000809AA2 AF68D8CD01000000000000000010000000BB62690EDF0A8F40 A2F9BFB776EB68A3
    And then :

    2013-08-21 12:15:32,980 WARN [LmtpServer-2] [name=user@domain.com;mid=87;ip=192.168.0.xxx;] ParsedMessage - Unable to parse part=2.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.2 0 filename= content-type=message/rfc822 message-id=<1335586568.87465.1377083725979.JavaMail.root@d omain.com>
    ...
    2013-08-21 14:34:16,080 WARN [LmtpServer-2] [name=user@domain.com;mid=87;ip=192.168.0.xxx;] ParsedMessage - Unable to parse part=2.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.2 2 filename= content-type=message/rfc822 message-id=<909660143.88512.1377020050080.JavaMail.root@do main.com>
    com.zimbra.cs.mime.MimeHandlerException: No content
    at com.zimbra.cs.mime.handler.MessageRFC822Handler.ge tContentImpl(MessageRFC822Handler.java:73)
    at com.zimbra.cs.mime.MimeHandler.getContent(MimeHand ler.java:157)
    at com.zimbra.cs.mime.ParsedMessage.analyzePart(Parse dMessage.java:1060)
    at com.zimbra.cs.mime.ParsedMessage.analyzeNonBodyPar ts(ParsedMessage.java:420)
    at com.zimbra.cs.mime.ParsedMessage.analyzeFully(Pars edMessage.java:454)
    at com.zimbra.cs.mailbox.CalendarItem.getIndexDocumen ts(CalendarItem.java:380)
    at com.zimbra.cs.mailbox.CalendarItem.generateIndexDa ta(CalendarItem.java:258)
    at com.zimbra.cs.mailbox.IndexHelper.indexItemList(In dexHelper.java:828)
    at com.zimbra.cs.mailbox.IndexHelper.indexDeferredIte msInternal(IndexHelper.java:354)
    at com.zimbra.cs.mailbox.IndexHelper.indexDeferredIte ms(IndexHelper.java:274)
    at com.zimbra.cs.mailbox.IndexHelper.maybeIndexDeferr edItems(IndexHelper.java:230)
    at com.zimbra.cs.mailbox.Mailbox.addMessage(Mailbox.j ava:4697)
    at com.zimbra.cs.mailbox.Mailbox.addMessage(Mailbox.j ava:4690)
    at com.zimbra.cs.mailbox.Mailbox.addMessage(Mailbox.j ava:4638)
    at com.zimbra.cs.filter.IncomingMessageHandler.addMes sage(IncomingMessageHandler.java:122)
    at com.zimbra.cs.filter.IncomingMessageHandler.implic itKeep(IncomingMessageHandler.java:116)
    at com.zimbra.cs.filter.ZimbraMailAdapter.doDefaultFi ling(ZimbraMailAdapter.java:343)
    at com.zimbra.cs.filter.RuleManager.applyRulesToIncom ingMessage(RuleManager.java:416)
    at com.zimbra.cs.filter.RuleManager.applyRulesToIncom ingMessage(RuleManager.java:363)
    at com.zimbra.cs.lmtpserver.ZimbraLmtpBackend.deliver MessageToLocalMailboxes(ZimbraLmtpBackend.java:571 )
    at com.zimbra.cs.lmtpserver.ZimbraLmtpBackend.deliver (ZimbraLmtpBackend.java:370)
    at com.zimbra.cs.lmtpserver.LmtpHandler.processMessag eData(LmtpHandler.java:367)
    at com.zimbra.cs.lmtpserver.TcpLmtpHandler.continueDA TA(TcpLmtpHandler.java:72)
    at com.zimbra.cs.lmtpserver.LmtpHandler.doDATA(LmtpHa ndler.java:356)
    at com.zimbra.cs.lmtpserver.LmtpHandler.processComman d(LmtpHandler.java:172)
    at com.zimbra.cs.lmtpserver.TcpLmtpHandler.processCom mand(TcpLmtpHandler.java:65)
    at com.zimbra.cs.tcpserver.ProtocolHandler.processCon nection(ProtocolHandler.java:199)
    at com.zimbra.cs.tcpserver.ProtocolHandler.run(Protoc olHandler.java:140)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run Task(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.io.IOException: No content
    at javax.mail.internet.MimePartDataSource.getInputStr eam(MimePartDataSource.java:108)
    at com.zimbra.cs.mime.handler.MessageRFC822Handler.ge tContentImpl(MessageRFC822Handler.java:57)
    ... 30 more
    Can u help me ?

Page 1 of 2 12 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Out of memory after TNEF message mapping
    By garg654 in forum Administrators
    Replies: 1
    Last Post: 05-21-2013, 12:10 PM
  2. Imap system folders mapping
    By pquerrec in forum Installation Help
    Replies: 0
    Last Post: 10-13-2012, 04:33 AM
  3. FATAL system - out of memory.
    By soltecsis in forum Administrators
    Replies: 4
    Last Post: 01-04-2012, 12:34 PM
  4. ZCS 7.1.3 consumed a lot of memory (all system memory)
    By soba@ukw.edu.pl in forum Administrators
    Replies: 9
    Last Post: 12-07-2011, 05:23 PM
  5. imapsync & tnef issues
    By Aderium in forum Migration
    Replies: 5
    Last Post: 04-14-2007, 12:32 AM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •