Results 1 to 10 of 10

Thread: Unexpected shutdown

  1. #1
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    7

    Default Unexpected shutdown

    Hi.
    Our installation has been working fine for a few months now. However twice in the last week it has completely shutdown. zmcontrol start gets everything up and running again with no problem, however I need to stop this happening.
    We are running the open sourced edition on ubuntu6.06, 2.66Ghz, 3GB RAM, 500GB hard disk.

    Is this a hardware problem?

    Catalina.out says:

    Code:
    2008-04-14 13:03:02
    Full thread dump Java HotSpot(TM) Client VM (1.6.0_02-b05 mixed mode):
    
    "Timer-11" daemon prio=10 tid=0x0849bc00 nid=0x7fbe in Object.wait() [0x4ef68000..0x4ef68fb0]
       java.lang.Thread.State: WAITING (on object monitor)
    	at java.lang.Object.wait(Native Method)
    	- waiting on <0x56921698> (a java.util.TaskQueue)
    	at java.lang.Object.wait(Object.java:485)
    	at java.util.TimerThread.mainLoop(Timer.java:483)
    	- locked <0x56921698> (a java.util.TaskQueue)
    	at java.util.TimerThread.run(Timer.java:462)
    
    "Timer-10" daemon prio=10 tid=0x0819e000 nid=0x561c in Object.wait() [0x525c9000..0x525ca130]
       java.lang.Thread.State: WAITING (on object monitor)
    	at java.lang.Object.wait(Native Method)
    	- waiting on <0x6c5052d8> (a java.util.TaskQueue)
    	at java.lang.Object.wait(Object.java:485)
    	at java.util.TimerThread.mainLoop(Timer.java:483)
    	- locked <0x6c5052d8> (a java.util.TaskQueue)
    	at java.util.TimerThread.run(Timer.java:462)
    
    "TcpWatchdog-1" daemon prio=10 tid=0x0871f400 nid=0x6dc8 in Object.wait() [0x52445000..0x52445f30]
       java.lang.Thread.State: WAITING (on object monitor)
    	at java.lang.Object.wait(Native Method)
    	- waiting on <0x737a90c0> (a EDU.oswego.cs.dl.util.concurrent.ClockDaemon)
    	at java.lang.Object.wait(Object.java:485)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon.nextTask(Unknown Source)
    	- locked <0x737a90c0> (a EDU.oswego.cs.dl.util.concurrent.ClockDaemon)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(Unknown Source)
    	at java.lang.Thread.run(Thread.java:619)
    
    "Timer-4" daemon prio=10 tid=0x4f73cc00 nid=0x5cd3 in Object.wait() [0x4f889000..0x4f88a130]
       java.lang.Thread.State: WAITING (on object monitor)
    	at java.lang.Object.wait(Native Method)
    	- waiting on <0x735e1730> (a java.util.TaskQueue)
    	at java.lang.Object.wait(Object.java:485)
    	at java.util.TimerThread.mainLoop(Timer.java:483)
    	- locked <0x735e1730> (a java.util.TaskQueue)
    	at java.util.TimerThread.run(Timer.java:462)
    
    "Thread-300" daemon prio=10 tid=0x4f7d6400 nid=0x5cd2 waiting on condition [0x4f98c000..0x4f98d1b0]
       java.lang.Thread.State: WAITING (parking)
    	at sun.misc.Unsafe.park(Native Method)
    	- parking to wait for  <0x735e17b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
    	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
    	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
    	at com.zimbra.cs.im.IMRouter.run(IMRouter.java:133)
    	at java.lang.Thread.run(Thread.java:619)
    
    "http-7071-Processor50" daemon prio=10 tid=0x0865d800 nid=0x171d in Object.wait() [0x4f9cd000..0x4f9ce130]
       java.lang.Thread.State: WAITING (on object monitor)
    	at java.lang.Object.wait(Native Method)
    	- waiting on <0x73032820> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
    	at java.lang.Object.wait(Object.java:485)
    	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
    	- locked <0x73032820> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
    	at java.lang.Thread.run(Thread.java:619)
    
    "http-7071-Processor49" daemon prio=10 tid=0x0865c400 nid=0x171c runnable [0x4fa0e000..0x4fa0f1b0]
       java.lang.Thread.State: RUNNABLE
    	at java.net.PlainSocketImpl.socketAccept(Native Method)
    	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
    	- locked <0x72c15c18> (a java.net.SocksSocketImpl)
    	at java.net.ServerSocket.implAccept(ServerSocket.java:453)
    	at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:259)
    	at org.apache.tomcat.util.net.jsse.JSSESocketFactory.acceptSocket(JSSESocketFactory.java:110)
    	at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:407)
    	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:70)
    	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
    	at java.lang.Thread.run(Thread.java:619)
    then a whole bunch more of similar output.
    and finally:

    Code:
    "VM Thread" prio=10 tid=0x080a5000 nid=0x15e2 runnable 
    
    "VM Periodic Task Thread" prio=10 tid=0x080d5c00 nid=0x15f4 waiting on condition 
    
    JNI global references: 1189
    
    Heap
     def new generation   total 465472K, used 184121K [0x530f0000, 0x72a00000, 0x72a40000)
      eden space 413760K,  40% used [0x530f0000, 0x5d4f38f8, 0x6c500000)
      from space 51712K,  31% used [0x6c500000, 0x6d4cadb0, 0x6f780000)
      to   space 51712K,   0% used [0x6f780000, 0x6f780000, 0x72a00000)
     tenured generation   total 1034240K, used 26964K [0x72a40000, 0xb1c40000, 0xb1cf0000)
       the space 1034240K,   2% used [0x72a40000, 0x744950a8, 0x74495200, 0xb1c40000)
     compacting perm gen  total 24320K, used 24147K [0xb1cf0000, 0xb34b0000, 0xb5cf0000)
       the space 24320K,  99% used [0xb1cf0000, 0xb3484fe0, 0xb3485000, 0xb34b0000)
    No shared spaces configured.
    
    [tomcat launcher] got signal SIGTERM invoking stop
    14-Apr-2008 13:03:04 org.apache.coyote.http11.Http11BaseProtocol pause
    INFO: Pausing Coyote HTTP/1.1 on http-80
    14-Apr-2008 13:03:04 org.apache.coyote.http11.Http11BaseProtocol pause
    INFO: Pausing Coyote HTTP/1.1 on http-7071
    14-Apr-2008 13:03:05 org.apache.catalina.core.StandardService stop
    INFO: Stopping service Catalina
    WARNING: Cannot load DocumentFactory: null
    14-Apr-2008 13:03:06 org.apache.coyote.http11.Http11BaseProtocol destroy
    INFO: Stopping Coyote HTTP/1.1 on http-80
    14-Apr-2008 13:03:06 org.apache.coyote.http11.Http11BaseProtocol destroy
    INFO: Stopping Coyote HTTP/1.1 on http-7071
    Any ideas?

  2. #2
    mmorse's Avatar
    mmorse is offline Moderator
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    20

    Default

    Off the top of my head no clue on that error, but wanted to confirm if that's 4.5.10_GA_1575.UBUNTU6? (from a prior post)
    http://www.zimbra.com/forums/announc...html#post62754
    -BTW 4.5.11 & 5.0.4 are out.

  3. #3
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    7

    Default

    zcs-4.5.10_GA_1575.UBUNTU6 is right.

    We are waiting for another server before we upgrade to the latest version.

    S

  4. #4
    jholder's Avatar
    jholder is offline Former Zimbran
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    20

    Default

    Need the whole log file.

  5. #5
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    7

    Default

    Attached is the catalina.out log for the day in question. Do you need any other logs?
    Attached Files Attached Files

  6. #6
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    7

    Default

    Our hosting company lost all connectivity with their dedicated servers yesterday :-(

    I spoke to them and at the exact time of our first outage last week they had some problem with the dns caching nameservers that may have led to problems with name resolution on our box.

    They have nothing logged at the time of the incident I describe above but it seems likely that it could have been the same problem.

    I attach the log from the time of the dns outage.

    Is there anything I can do to protect against this in the future or is it just an external problem that I cannot do anything about?

    Thanks,
    S
    Attached Files Attached Files

  7. #7
    jholder's Avatar
    jholder is offline Former Zimbran
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    20

    Default

    hmm
    Looks to me like you ran out of threads. How many IMAP clients do you have?

    Lots of locked threads:
    - locked <0x56921698> (a java.util.TaskQueue)
    - locked <0x6c5052d8> (a java.util.TaskQueue)
    - locked <0x737a90c0> (a EDU.oswego.cs.dl.util.concurrent.ClockDaemon)
    - locked <0x735e1730> (a java.util.TaskQueue)
    - locked <0x73032820> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x72c15c18> (a java.net.SocksSocketImpl)
    - locked <0x73032340> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730320d0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73031e60> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73031bf0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73031980> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73031710> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730314a0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73031230> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030fc0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030d50> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030ae0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030870> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030478> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030208> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ff98> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302fd28> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302fab8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302f848> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302f5d8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302f368> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302f0f8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ee88> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ec18> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302e9a8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302e738> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302e4c8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302e258> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302dfe8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302dd78> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302db08> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302d898> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302d628> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302d3b8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302d148> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ced8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302cc68> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302c9f8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302c788> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302c518> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302be98> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302bc28> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302b9b8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302b748> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302b4d8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302b268> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302aff8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ad88> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ab18> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301cd98> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301cb28> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301c8b8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301c648> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301c3d8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301c168> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301bef8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x72b4d108> (a java.net.SocksSocketImpl)
    - locked <0x7301ba18> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301b7a8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301b538> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301b2c8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301b058> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301ade8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301ab78> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301a908> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301a698> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301a428> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301a1b8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73019f48> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73019cd8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73019a68> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730197f8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73019588> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73019318> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730190a8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73018e38> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73018bc8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73018650> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730183e0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73018170> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73017f00> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73017c90> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73017a20> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730177b0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73017540> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730172d0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73017060> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73016df0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73016b80> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73016910> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730166a0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73016430> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730161c0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015f50> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015ce0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015a70> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015800> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015590> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015320> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730150b0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73014e40> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73014bd0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73014960> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730146f0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73014480> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73014210> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013fa0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013d30> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013ac0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013850> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730135e0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013370> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013100> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73012e90> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73012c20> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730129b0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73012740> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730124d0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73012260> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73011ff0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73011d80> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73011b10> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730118a0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73011630> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730113c0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73011150> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010ee0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010c70> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010a00> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010790> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010520> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730102b0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010040> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300fdd0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300fb60> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300f8f0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300f680> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300f410> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300f1a0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300ef30> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300ecc8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300ea60> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300e7f8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300e590> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300e328> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300e0c0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300de58> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300dbf0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300d988> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7304dc08> (a com.zimbra.cs.store.FileBlobStore$IncomingDirector ySweeper)
    - locked <0x72ed7d68> (a java.net.SocksSocketImpl)
    - locked <0x72ed7be0> (a java.net.SocksSocketImpl)
    - locked <0x72ed7a68> (a java.net.SocksSocketImpl)
    - locked <0x72ed5ed0> (a java.net.SocksSocketImpl)
    - locked <0x72fa71a0> (a java.net.SocksSocketImpl)
    - locked <0x72f98338> (a sun.nio.ch.Util$1)
    - locked <0x72f98328> (a java.util.Collections$UnmodifiableSet)
    - locked <0x72f97fe0> (a sun.nio.ch.EPollSelectorImpl)
    - locked <0x72f93250> (a com.zimbra.cs.index.MailboxIndex$IndexWritersSweep er)
    - locked <0x72f22f40> (a java.util.TaskQueue)
    - locked <0x72f0d430> (a java.util.TaskQueue)
    - locked <0x72e3a3c8> (a com.sun.jndi.ldap.pool.PoolCleaner)
    - locked <0x72a40a80> (a java.lang.ref.ReferenceQueue$Lock)
    - locked <0x72a40980> (a java.lang.ref.Reference$Lock)
    Can you take a look at your mysql log and your mailbox.log and look for problems. Maybe a condition happened that caused it to run out of worker threads.

  8. #8
    jholder's Avatar
    jholder is offline Former Zimbran
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    20

    Default

    Quote Originally Posted by geoffDeGeoffGeoff View Post
    Is there anything I can do to protect against this in the future or is it just an external problem that I cannot do anything about?
    Yeah, i've actually heard of dns failures causing issues. One way to get around it is to run DNS in house, and have it point to your hosting providers as a external resolver. That way if it goes down, your internal will keep working.

  9. #9
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    7

    Default

    So do you think that running out of threads was related to the dns problem?

    By 'IMAP client' do you mean the number of people using a mail client to connect to our server using IMAP? If so then there cannot be more than a handful, most are using the web client and we have less than a hundred accounts that are actually in use.

  10. #10
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    7

    Default

    Here is an extract from mailbox.log at the time of the dns outage:
    Code:
    2008-04-11 00:00:19,107 WARN  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefOpenMailInNewWindow
    2008-04-11 00:00:19,107 WARN  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefSentLifetime
    2008-04-11 00:00:19,107 WARN  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefTrashLifetime
    2008-04-11 00:00:19,107 WARN  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefJunkLifetime
    2008-04-11 00:00:19,107 WARN  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefInboxUnreadLifetime
    2008-04-11 00:00:19,110 INFO  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] SoapEngine - handler exception
    com.zimbra.cs.account.AccountServiceException: invalid attr name: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]
    	at com.zimbra.cs.account.AccountServiceException.INVALID_ATTR_NAME(AccountServiceException.java:105)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:289)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:268)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:248)
    	at com.zimbra.cs.service.account.ModifyPrefs.handle(ModifyPrefs.java:60)
    	at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:266)
    	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:163)
    	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:85)
    	at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:220)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
    	at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:152)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
    	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
    	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
    	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
    	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
    	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
    	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
    	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
    	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667)
    	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
    	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
    	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
    	at java.lang.Thread.run(Thread.java:619)
    Caused by: javax.naming.directory.InvalidAttributeIdentifierException: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]; remaining name 'uid=iainkb,ou=people,dc=cmhmail,dc=com'
    	at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3074)
    	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2951)
    	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2758)
    	at com.sun.jndi.ldap.LdapCtx.c_modifyAttributes(LdapCtx.java:1441)
    	at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_modifyAttributes(ComponentDirContext.java:255)
    	at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.modifyAttributes(PartialCompositeDirContext.java:172)
    	at javax.naming.directory.InitialDirContext.modifyAttributes(InitialDirContext.java:153)
    	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttributes(LdapUtil.java:1052)
    	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttrs(LdapUtil.java:590)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:286)
    	... 25 more
    2008-04-11 00:01:00,892 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefOpenMailInNewWindow
    2008-04-11 00:01:00,893 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefSentLifetime
    2008-04-11 00:01:00,893 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefTrashLifetime
    2008-04-11 00:01:00,893 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefJunkLifetime
    2008-04-11 00:01:00,893 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefInboxUnreadLifetime
    2008-04-11 00:01:00,895 INFO  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] SoapEngine - handler exception
    com.zimbra.cs.account.AccountServiceException: invalid attr name: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]
    	at com.zimbra.cs.account.AccountServiceException.INVALID_ATTR_NAME(AccountServiceException.java:105)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:289)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:268)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:248)
    	at com.zimbra.cs.service.account.ModifyPrefs.handle(ModifyPrefs.java:60)
    	at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:266)
    	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:163)
    	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:85)
    	at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:220)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
    	at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:152)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
    	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
    	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
    	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
    	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
    	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
    	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
    	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
    	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667)
    	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
    	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
    	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
    	at java.lang.Thread.run(Thread.java:619)
    Caused by: javax.naming.directory.InvalidAttributeIdentifierException: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]; remaining name 'uid=iainkb,ou=people,dc=cmhmail,dc=com'
    	at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3074)
    	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2951)
    	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2758)
    	at com.sun.jndi.ldap.LdapCtx.c_modifyAttributes(LdapCtx.java:1441)
    	at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_modifyAttributes(ComponentDirContext.java:255)
    	at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.modifyAttributes(PartialCompositeDirContext.java:172)
    	at javax.naming.directory.InitialDirContext.modifyAttributes(InitialDirContext.java:153)
    	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttributes(LdapUtil.java:1052)
    	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttrs(LdapUtil.java:590)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:286)
    	... 25 more
    2008-04-11 00:01:41,916 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefOpenMailInNewWindow
    2008-04-11 00:01:41,916 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefSentLifetime
    2008-04-11 00:01:41,916 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefTrashLifetime
    2008-04-11 00:01:41,916 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefJunkLifetime
    2008-04-11 00:01:41,916 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefInboxUnreadLifetime
    2008-04-11 00:01:41,919 INFO  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] SoapEngine - handler exception
    com.zimbra.cs.account.AccountServiceException: invalid attr name: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]
    	at com.zimbra.cs.account.AccountServiceException.INVALID_ATTR_NAME(AccountServiceException.java:105)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:289)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:268)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:248)
    	at com.zimbra.cs.service.account.ModifyPrefs.handle(ModifyPrefs.java:60)
    	at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:266)
    	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:163)
    	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:85)
    	at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:220)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
    	at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:152)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
    	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
    	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
    	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
    	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
    	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
    	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
    	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
    	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667)
    	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
    	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
    	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
    	at java.lang.Thread.run(Thread.java:619)
    Caused by: javax.naming.directory.InvalidAttributeIdentifierException: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]; remaining name 'uid=iainkb,ou=people,dc=cmhmail,dc=com'
    	at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3074)
    	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2951)
    	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2758)
    	at com.sun.jndi.ldap.LdapCtx.c_modifyAttributes(LdapCtx.java:1441)
    	at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_modifyAttributes(ComponentDirContext.java:255)
    	at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.modifyAttributes(PartialCompositeDirContext.java:172)
    	at javax.naming.directory.InitialDirContext.modifyAttributes(InitialDirContext.java:153)
    	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttributes(LdapUtil.java:1052)
    	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttrs(LdapUtil.java:590)
    	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:286)
    	... 25 more
    Not really my area but this looks like some thread guzzling as a result of being unable to do something for one client. Probably the dns problem?

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Zimbra fails after working for 2 weeks
    By Linsys in forum Administrators
    Replies: 10
    Last Post: 10-07-2008, 12:42 AM
  2. POP3 server dies every hour
    By jdossey7 in forum Administrators
    Replies: 12
    Last Post: 09-27-2007, 01:14 PM
  3. Clean Shutdown of Zimbra
    By swu in forum Administrators
    Replies: 1
    Last Post: 07-14-2007, 05:28 AM
  4. Replies: 29
    Last Post: 03-06-2007, 01:14 PM
  5. Replies: 0
    Last Post: 09-23-2006, 10:25 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •