java.lang.OutOfMemoryError while performing import of 10k messages via zmmailbox

[jpatrick281]

I have a mailbox which I'm trying to import using zmmailbox batch commands as part of a larger migration process which I can continually replicate an outofmemory error on.

Here's the box:

/home/user/cur
[root@ztest cur]# find | wc -l
10130
[root@ztest cur]#

Here's the import command:
[root@ztest cur]# /opt/zimbra/bin/zmmailbox -d -z -m ppelanne@domain.com addMessage Inbox cur/

Here's where it dies:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
<soap:Header>
<context xmlns="urn:zimbra">
<session id="1437">1437</session>
<change token="7194"/>
<notify seq="1391">
<deleted id="-1744"/>
<created>
<m id="8008" d="1309438521000" rev="7194" s="2578" l="2" cid="8009">
<e d="Nathan" t="f" a="alerts@somedomain.com" p="Someperson"/>
<su>[Austin-exceptions] Someperson - Day - Admin Training - Tardy - 06/30/11</su>
<fr>Shift: Day Schedule Pattern: 8AM - 5PM (Tues -Sat) Call ID: Reason: Is running about 15 minutes late in traffic. Will get with his Supervisor ...</fr>
</m>
<c id="8009" d="1309438521000" n="2">
<su>[Austin-exceptions] Someguy - Day - Admin Training - Tardy - 06/30/11</su>
<e d="Nathan" t="f" a="alerts@somedomain.com" p="Someperson"/>
</c>
</created>
<modified>
<folder id="2" s="321049290" i4next="8009" i4ms="7194" n="7177"/>
<m id="1744" cid="8009"/>
<mbx s="321306894"/>
</modified>
</notify>
</context>
</soap:Header>
<soap:Body>
<AddMsgResponse xmlns="urn:zimbraMail">
<m f="" id="8008" d="1309438521000" rev="7194" t="" s="2578" md="1310498060" ms="7194" l="2" cid="8009"/>
</AddMsgResponse>
</soap:Body>
</soap:Envelope>
=============================== (13 msecs)
8008 (cur/1309438518.H579253P3581.somedomain.com,S=2578:2,)
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at com.zimbra.common.util.BufferStream.buffer(BufferS tream.java:72)
at com.zimbra.common.util.BufferStream.readFrom(Buffe rStream.java:199)
at com.zimbra.common.util.ByteUtil.getContent(ByteUti l.java:185)
at com.zimbra.common.util.ByteUtil.getContent(ByteUti l.java:159)
at com.zimbra.common.util.ByteUtil.getContent(ByteUti l.java:135)
at com.zimbra.common.mime.MimeMessage.<init>(MimeMess age.java:126)
at com.zimbra.common.mime.shim.JavaMailMimeMessage.<i nit>(JavaMailMimeMessage.java:76)
at com.zimbra.cs.zclient.ZMailboxUtil.addMessage(ZMai lboxUtil.java:1760)
at com.zimbra.cs.zclient.ZMailboxUtil.doAddMessage(ZM ailboxUtil.java:1785)
at com.zimbra.cs.zclient.ZMailboxUtil.execute(ZMailbo xUtil.java:916)
at com.zimbra.cs.zclient.ZMailboxUtil.main(ZMailboxUt il.java:2679)
[root@ztest ppelanne]#



I've done a bunch of researching and I've seen some people speak about raising the mailboxd_java_heap_memory_percent variable and I've raised it to 75 with no results (it was previously NULL).

[root@ztest cur]# zmlocalconfig mailboxd_java_heap_memory_percent
mailboxd_java_heap_memory_percent = 75
[root@ztest cur]#

Any help would be greatly appreciated.

[jpatrick281]

EDIT: The workaround I found is far too slow for the amount of mail I need to import. Any help on this would be greatly appreciated.

[jpatrick281]

Still experiencing this issue, any ideas?

[jpatrick281]

Solved this issue by running the command line manually instead of using zmmailbox and changing -Xmx256m to -Xmx1024m

Like so:

[root@ztest cur]# /opt/zimbra/java/bin/java -XX:ErrorFile=/opt/zimbra/log -client -Xmx1024m -Dzimbra.home=/opt/zimbra -Djava.library.path=/opt/zimbra/lib -Djava.ext.dirs=/opt/zimbra/java/jre/lib/ext:/opt/zimbra/lib/jars:/opt/zimbra/lib/ext-common:/opt/zimbra/lib/ext/clamscanner com.zimbra.cs.zclient.ZMailboxUtil -z -m ppelanne@somedomain.com addMessage Inbox .

Edit: Just a note, the command above is setup to be run inside of the directory you wish to import in a maildir structure. In this case the cwd was: /home/mail/user/cur