We've been on a single server for some time now and we'd like to move to a multi-server setup. Right now we have 3 new zimbra servers 1 ldap, 1 imap/pop/http proxy, and 1 mailbox server. I'm trying to use postRestURL to copy users from our current single server to the new multi-server environment. After moving everyone I'll reinstall add the current server as a second mailbox server.
The new servers seemed to be working fine but when I try to copy a user over with postRestURL I'm getting timeout errors after only a few hundred emails are copied. Here is the verbose error from the command:
Code:
[zimbra@mailboxhost ~]$ zmmailbox -v -z -m test_user@domain.com postRestURL '//?fmt=tgz' /tmp/user.tgz
ERROR: zclient.IO_ERROR (Read timed out) (cause: java.net.SocketTimeoutException Read timed out)
com.zimbra.cs.zclient.ZClientException: Read timed out
ExceptionId:main:1255976953128:bf17ac7b9587da1a
Code:zclient.IO_ERROR
at com.zimbra.cs.zclient.ZClientException.IO_ERROR(ZClientException.java:45)
at com.zimbra.cs.zclient.ZMailbox.postRESTResource(ZMailbox.java:2650)
at com.zimbra.cs.zclient.ZMailboxUtil.doPostRestURL(ZMailboxUtil.java:2692)
at com.zimbra.cs.zclient.ZMailboxUtil.execute(ZMailboxUtil.java:1115)
at com.zimbra.cs.zclient.ZMailboxUtil.main(ZMailboxUtil.java:2575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.zimbra.common.util.ConsoleRunner.main(ConsoleRunner.java:96)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:789)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:746)
at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:77)
at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:105)
at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1115)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1373)
at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1832)
at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1590)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:995)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
at com.zimbra.cs.zclient.ZMailbox.postRESTResource(ZMailbox.java:2642)
... 8 more
From mailbox.log I see:
Code:
2009-10-19 14:28:07,704 INFO [btpool0-6] [name=zimbra;ip=127.0.0.1;] misc - delegated access: doc=DelegateAuth, authenticated account=zimbra, target account=test_user@domain.com
2009-10-19 14:28:07,780 INFO [btpool0-6] [name=test_user@domain.com;aname=zimbra;mid=33;ip=mailbox_server's IP;ua=zclient/6.0.1_GA_1816.RHEL5_64;] soap - GetInfoRequest
2009-10-19 14:28:07,781 INFO [btpool0-6] [name=test_user@domain.com;aname=zimbra;mid=33;ip=mailbox_server's IP;ua=zclient/6.0.1_GA_1816.RHEL5_64;] misc - delegated access: doc=GetInfo, delegating account=zimbra, authenticated account=test_user@domain.com, target account=test_user@domain.com
2009-10-19 14:28:13,142 INFO [btpool0-6] [name=test_user@domain.com;mid=33;ip=proxyhost's IP;] mailbox - UserServlet (POST): http://proxyhost.domain.com/service/home/test_user@domain.com/
2009-10-19 14:28:13,143 INFO [btpool0-6] [name=test_user@domain.com;mid=33;ip=159.28.1.123;] mailbox - UserServlet received file unknown - 255036533 request bytes
2009-10-19 14:28:15,396 INFO [btpool0-6] [name=test_user@domain.com;mid=33;ip=proxyhost's IP;] mailop - adding folder zimbra scripts: id=260, parentId=16.
2009-10-19 14:28:15,597 INFO [btpool0-6] [name=test_user@domain.com;mid=33;ip=proxyhost's IP;] mailop - adding folder ECS - private: id=261, parentId=1.
<snip>
2009-10-19 14:28:36,281 INFO [btpool0-6] [name=test_user@domain.com;mid=33;ip=159.28.1.123;] mailop - Adding Message: id=385, Message-ID=<200812151758.mBFHwtiW024458@hostname.domain.com>, parentId=-1, folderId=2, folderName=Inbox.
2009-10-19 14:28:36,538 INFO [btpool0-6] [name=test_user@domain.com;mid=33;ip=159.28.1.123;] mailop - Adding Message: id=386, Message-ID=<327B695B-5894-4417-AB19-1516CAFC91B7@domain.com>, parentId=-1, folderId=2, folderName=Inbox.
<snip>
2009-10-19 14:29:12,181 INFO [btpool0-6] [name=test_user@domain.com;mid=33;ip=159.28.1.123;] mailop - Adding Message: id=473, Message-ID=<B202FFF7-1480-47D0-B648-136D754B88DA@domain.com>, parentId=-1, folderId=2, folderName=Inbox.
2009-10-19 14:29:42,343 WARN [btpool0-6] [name=test_user@domain.com;mid=33;ip=159.28.1.123;] misc - timeout: path=/Inbox/
2009-10-19 14:29:42,344 ERROR [btpool0-6] [] log - /service/home/test_user@domain.com/
org.mortbay.io.RuntimeIOException: org.mortbay.jetty.EofException
at org.mortbay.io.UncheckedPrintWriter.newLine(UncheckedPrintWriter.java:246)
at org.mortbay.io.UncheckedPrintWriter.println(UncheckedPrintWriter.java:398)
at com.zimbra.cs.service.formatter.Formatter.updateClient(Formatter.java:248)
at com.zimbra.cs.service.formatter.Formatter.updateClient(Formatter.java:293)
at com.zimbra.cs.service.formatter.Formatter.updateClient(Formatter.java:258)
at com.zimbra.cs.service.formatter.Formatter.save(Formatter.java:104)
at com.zimbra.cs.service.UserServlet.doPost(UserServlet.java:616)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:187)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1148)
at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:81)
at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:145)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1139)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:378)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.handler.rewrite.RewriteHandler.handle(RewriteHandler.java:230)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:324)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:517)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:868)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:386)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)
Caused by: org.mortbay.jetty.EofException
at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:631)
at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:579)
at java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java:109)
at org.mortbay.jetty.AbstractGenerator$OutputWriter.write(AbstractGenerator.java:903)
at org.mortbay.jetty.AbstractGenerator$OutputWriter.write(AbstractGenerator.java:752)
at java.io.Writer.write(Writer.java:140)
at org.mortbay.io.UncheckedPrintWriter.newLine(UncheckedPrintWriter.java:235)
... 32 more
On the proxy server in the log I see:
Code:
2009/10/19 14:28:07 [warn] 26592#0: *59 a client request body is buffered to a temporary file /opt/zimbra/nginx-0.5.37-zimbra/client_body_temp/0000000002, client: mailbox's IP, server: proxyhost.domain.com, request: "POST /home/test_user@domain.com?fmt=tgz HTTP/1.1", host: "proxyhost.domain.com"
2009/10/19 14:29:42 [info] 26592#0: *59 client 159.28.1.234 closed keepalive connection
I've used getRestURL/postRestURL many times on our single server configurations without problems so I was wondering if this is an issue with this using the web proxy? Has anyone else seen this problem before?