Results 1 to 6 of 6

Thread: [SOLVED] ZCS hang for 10 minutes when sending e-mail with a large number of recipient

  1. #1
    ChaSer is offline Loyal Member
    Join Date
    Oct 2009
    Location
    Ukraine
    Posts
    79
    Rep Power
    5

    Default [SOLVED] ZCS hang for 10 minutes when sending e-mail with a large number of recipient

    ZCS hang for 10 minutes when somebody sending one e-mail with a large number (95 in my case) of recipients from internal domain.
    This html e-mail message has very small size: contains 5 string of text with links to document published in Alfresco and its folder, no pictures or attachments.
    Some recipients receive this message soon (in few seconds), but othes can not connect to the server through POP/IMAP/Zimbra/Web-console for 10 minutes.
    Others receive this message after 10 minutes.
    The result of "top" command at this time looks good: no proccessor load, no swap used, 150Mb RAM free, no disks wait etc.
    "zmcontrol status" reply quickly and show all processes running.
    Telnet to SMTP reply with Postfix greeting in the twinkling of an eye, but telnet to POP or IMAP not connects - hangs.
    Server configuration for 190 users:
    AMD Athlon 64 X2 5000+
    DDR2 4Gb
    2xHDD SATAII 500Gb (program RAID1)

    In Zimbra web admin console I get errors like:
    In "User accounts": Msg: system failure: ZimbraLdapContext Code: service.FAILURE Method: SearchDirectoryRequest Detail:soap:Receiver
    In "Domains": Msg: system failure: ZimbraLdapContext Code: service.FAILURE Method: GetAllZimletsRequest Detail:soap:Receiver
    In "Servers": Msg: system failure: ZimbraLdapContext Code: service.FAILURE Method: GetServerRequest Detail:soap:Receiver
    In "Mail queue": Msg: system failure: exception during auth {RemoteManager: mail.local->zimbra@mail.local:22} Code: service.FAILURE Method: GetMailQueueInfoRequest Detail:soap:Receiver
    etc.

    If there is enough hardware resources why ZCS hangs for some time ?
    Last edited by ChaSer; 09-14-2010 at 02:47 AM.

  2. #2
    ChaSer is offline Loyal Member
    Join Date
    Oct 2009
    Location
    Ukraine
    Posts
    79
    Rep Power
    5

    Default

    I done mailing for 2500 external recipients yesterday.
    The result of "top" command at this time looks good: proccessor load not exceed 80%, no swap used, 800Mb RAM free, no disks wait etc.
    ZCS hangs for 1.5 hours, but some times I can send message through web console and some times can see mail queue in web admin console.
    I send message to my external mailbox and get it in some seconds.
    I send message from my zimbra mailbox to itself and get it in 20 minutes.
    My ZCS server get about 200 "wrong email address" reply from recipients servers after send all 2500 messages.
    This 200 messages appear in pending mail queue and its number dont decrease for 10 minutes.
    Mail queue be in processing so I can not process it manually and be forced to reboot ZCS.
    Mail queue starts to processing after rebooting.
    So problem is somewhere in incoming mail processing.

    Where ? Maybe some configuration hints can help ?

  3. #3
    phoenix is online now Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,470
    Rep Power
    56

    Default

    Quote Originally Posted by ChaSer View Post
    Where ? Maybe some configuration hints can help ?
    You need to look in the log files and see what's happening on your server otherwise it's just guesswork.
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  4. #4
    ChaSer is offline Loyal Member
    Join Date
    Oct 2009
    Location
    Ukraine
    Posts
    79
    Rep Power
    5

    Default

    Quote Originally Posted by phoenix View Post
    You need to look in the log files and see what's happening on your server otherwise it's just guesswork.
    Does Your post meen I not overload the server and it should not freeze ?
    I set DEBUG mode but can not find something interesting in log files.
    Is that such from mailbox.log:

    2010-09-15 16:48:54,379 INFO [btpool0-7://mail.local:7071/service/admin/soap/GetInfoRequest] [name=myusername@mydomain.com;aname=zimbra;mid=76;i p=10.10.20.12;ua=zclient/6.0.8_GA_2673;] soap - GetInfoRequest
    2010-09-15 16:48:54,380 INFO [btpool0-7://mail.local:7071/service/admin/soap/GetInfoRequest] [name=myusername@mydomain.com;aname=zimbra;mid=76;i p=10.10.20.12;ua=zclient/6.0.8_GA_2673;] misc - delegated access: doc=GetInfo, delegating account=zimbra, authenticated account=myusername@mydomain.com, target account=myusername@mydomain.com
    2010-09-15 16:48:54,382 ERROR [btpool0-7://mail.local:7071/service/admin/soap/GetInfoRequest] [name=myusername@mydomain.com;aname=zimbra;mid=76;i p=10.10.20.12;ua=zclient/6.0.8_GA_2673;] account - can't get identities
    com.zimbra.common.service.ServiceException: system failure: ZimbraLdapContext
    ExceptionId:btpool0-7://mail.local:7071/service/admin/soap/GetInfoRequest:1284558534382:9a9263a23b22232b
    Code:service.FAILURE
    at com.zimbra.common.service.ServiceException.FAILURE (ServiceException.java:248)
    at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init >(ZimbraLdapContext.java:416)
    at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init >(ZimbraLdapContext.java:366)
    at com.zimbra.cs.account.ldap.LdapProvisioning.getIde ntitiesByQuery(LdapProvisioning.java:4993)
    at com.zimbra.cs.account.ldap.LdapProvisioning.getAll Identities(LdapProvisioning.java:5196)
    at com.zimbra.cs.service.account.GetInfo.doIdentities (GetInfo.java:246)
    at com.zimbra.cs.service.account.GetInfo.handle(GetIn fo.java:164)
    at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEng ine.java:420)
    at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:274)
    at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.jav a:158)
    at com.zimbra.soap.SoapServlet.doWork(SoapServlet.jav a:291)
    at com.zimbra.soap.SoapServlet.doPost(SoapServlet.jav a:212)
    at javax.servlet.http.HttpServlet.service(HttpServlet .java:727)
    at com.zimbra.cs.servlet.ZimbraServlet.service(Zimbra Servlet.java:181)
    at javax.servlet.http.HttpServlet.service(HttpServlet .java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(Ser vletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedCha in.doFilter(ServletHandler.java:1166)
    at com.zimbra.cs.servlet.SetHeaderFilter.doFilter(Set HeaderFilter.java:79)
    at org.mortbay.jetty.servlet.ServletHandler$CachedCha in.doFilter(ServletHandler.java:1157)
    at org.mortbay.servlet.UserAgentFilter.doFilter(UserA gentFilter.java:81)
    at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter .java:155)
    at org.mortbay.jetty.servlet.ServletHandler$CachedCha in.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(Se rvletHandler.java:388)
    at org.mortbay.jetty.security.SecurityHandler.handle( SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(Se ssionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(Co ntextHandler.java:765)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebA ppContext.java:418)
    at org.mortbay.jetty.handler.ContextHandlerCollection .handle(ContextHandlerCollection.java:230)
    at org.mortbay.jetty.handler.HandlerCollection.handle (HandlerCollection.java:114)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(Ha ndlerWrapper.java:152)
    at org.mortbay.jetty.handler.rewrite.RewriteHandler.h andle(RewriteHandler.java:230)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(Ha ndlerWrapper.java:152)
    at org.mortbay.jetty.handler.DebugHandler.handle(Debu gHandler.java:77)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(Ha ndlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(Htt pConnection.java:543)
    at org.mortbay.jetty.HttpConnection$RequestHandler.co ntent(HttpConnection.java:939)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser. java:755)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpPa rser.java:218)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnec tion.java:405)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(Selec tChannelEndPoint.java:413)
    at org.mortbay.thread.BoundedThreadPool$PoolThread.ru n(BoundedThreadPool.java:451)
    Caused by: javax.naming.CommunicationException: Bad file descriptor [Root exception is java.net.SocketException: Bad file descriptor]
    at com.sun.jndi.ldap.LdapCtx.extendedOperation(LdapCt x.java:3213)
    at javax.naming.ldap.InitialLdapContext.extendedOpera tion(InitialLdapContext.java:164)
    at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init >(ZimbraLdapContext.java:406)
    ... 40 more
    Caused by: java.net.SocketException: Bad file descriptor
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutp utStream.java:92)
    at java.net.SocketOutputStream.write(SocketOutputStre am.java:136)
    at java.io.BufferedOutputStream.flushBuffer(BufferedO utputStream.java:65)
    at java.io.BufferedOutputStream.flush(BufferedOutputS tream.java:123)
    at com.sun.jndi.ldap.Connection.writeRequest(Connecti on.java:396)
    at com.sun.jndi.ldap.LdapClient.extendedOp(LdapClient .java:1172)
    at com.sun.jndi.ldap.LdapCtx.extendedOperation(LdapCt x.java:3160)
    ... 42 more

  5. #5
    phoenix is online now Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,470
    Rep Power
    56

    Default

    Quote Originally Posted by ChaSer View Post
    Does Your post meen I not overload the server and it should not freeze ?
    I've no idea whether your server is overloaded or not as you've given no information about the hardware, is it a VM or real hardware, number of users, installed RAM, whether your user are mostly Web UI users or IMAP/POP3 clients, etc. etc... Everything is guesswork unless you give a full description of the problem, your configuration and the exact circumstances under which it occurs. Certainly your server should not be 'hanging' for 10 minutes so you can start by searching the wiki for the words 'performance' and 'zmstat' and review some of the articles on how to produce graphs to monitor your server.

    Quote Originally Posted by ChaSer View Post
    I set DEBUG mode but can not find something interesting in log files.
    Where did you set Debug mode?

    Quote Originally Posted by ChaSer View Post
    Is that such from mailbox.log:
    Start with this: Bug 42870 &ndash; "Bad file descriptor" SocketException causes system failure every several hours
    Last edited by phoenix; 12-31-2010 at 06:10 AM.
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  6. #6
    ChaSer is offline Loyal Member
    Join Date
    Oct 2009
    Location
    Ukraine
    Posts
    79
    Rep Power
    5

    Default

    You are THE BEST !

    My Ubuntu 10.04 max opened file limit is 382416 (cat /proc/sys/fs/file-max)
    So command "ulimit -n 524288" in zimbra user .bashrc (/opt/zimbra/.bashrc) file get error ant zimbra user max opened file limit was setted to 1024.
    Root limit was to 1024.

    I add in end of /etc/security/limits.conf:
    zimbra soft nofile 382416
    zimbra hard nofile 382416
    root soft nofile 382416
    root hard nofile 382416
    and just only restart ZCS by "zmcontrol restart".

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. zmmailboxd not running
    By jerkykid in forum Administrators
    Replies: 15
    Last Post: 02-17-2011, 11:02 PM
  2. Replies: 7
    Last Post: 02-03-2011, 07:01 AM
  3. [SOLVED] Zimbra OSS 6.04 will not start
    By powrrrplay in forum Installation
    Replies: 6
    Last Post: 01-25-2010, 07:15 PM
  4. No email in or out
    By ValueVPS in forum Installation
    Replies: 37
    Last Post: 01-13-2010, 12:56 PM
  5. [SOLVED] Mailserver down when send file attach of 50Mb
    By ZMilton in forum Administrators
    Replies: 20
    Last Post: 04-10-2008, 11:44 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •