Results 1 to 10 of 10

Thread: Unexpected shutdown

  1. 04-14-2008, 09:42 PM #1
    geoffDeGeoffGeoff
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    6

    Default Unexpected shutdown

    Hi.
    Our installation has been working fine for a few months now. However twice in the last week it has completely shutdown. zmcontrol start gets everything up and running again with no problem, however I need to stop this happening.
    We are running the open sourced edition on ubuntu6.06, 2.66Ghz, 3GB RAM, 500GB hard disk.

    Is this a hardware problem?

    Catalina.out says:

    Code:
    2008-04-14 13:03:02
Full thread dump Java HotSpot(TM) Client VM (1.6.0_02-b05 mixed mode):

"Timer-11" daemon prio=10 tid=0x0849bc00 nid=0x7fbe in Object.wait() [0x4ef68000..0x4ef68fb0]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x56921698> (a java.util.TaskQueue)
	at java.lang.Object.wait(Object.java:485)
	at java.util.TimerThread.mainLoop(Timer.java:483)
	- locked <0x56921698> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

"Timer-10" daemon prio=10 tid=0x0819e000 nid=0x561c in Object.wait() [0x525c9000..0x525ca130]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x6c5052d8> (a java.util.TaskQueue)
	at java.lang.Object.wait(Object.java:485)
	at java.util.TimerThread.mainLoop(Timer.java:483)
	- locked <0x6c5052d8> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

"TcpWatchdog-1" daemon prio=10 tid=0x0871f400 nid=0x6dc8 in Object.wait() [0x52445000..0x52445f30]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x737a90c0> (a EDU.oswego.cs.dl.util.concurrent.ClockDaemon)
	at java.lang.Object.wait(Object.java:485)
	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon.nextTask(Unknown Source)
	- locked <0x737a90c0> (a EDU.oswego.cs.dl.util.concurrent.ClockDaemon)
	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(Unknown Source)
	at java.lang.Thread.run(Thread.java:619)

"Timer-4" daemon prio=10 tid=0x4f73cc00 nid=0x5cd3 in Object.wait() [0x4f889000..0x4f88a130]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x735e1730> (a java.util.TaskQueue)
	at java.lang.Object.wait(Object.java:485)
	at java.util.TimerThread.mainLoop(Timer.java:483)
	- locked <0x735e1730> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

"Thread-300" daemon prio=10 tid=0x4f7d6400 nid=0x5cd2 waiting on condition [0x4f98c000..0x4f98d1b0]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x735e17b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
	at com.zimbra.cs.im.IMRouter.run(IMRouter.java:133)
	at java.lang.Thread.run(Thread.java:619)

"http-7071-Processor50" daemon prio=10 tid=0x0865d800 nid=0x171d in Object.wait() [0x4f9cd000..0x4f9ce130]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x73032820> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:485)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
	- locked <0x73032820> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:619)

"http-7071-Processor49" daemon prio=10 tid=0x0865c400 nid=0x171c runnable [0x4fa0e000..0x4fa0f1b0]
   java.lang.Thread.State: RUNNABLE
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
	- locked <0x72c15c18> (a java.net.SocksSocketImpl)
	at java.net.ServerSocket.implAccept(ServerSocket.java:453)
	at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:259)
	at org.apache.tomcat.util.net.jsse.JSSESocketFactory.acceptSocket(JSSESocketFactory.java:110)
	at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:407)
	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:70)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
	at java.lang.Thread.run(Thread.java:619)
    then a whole bunch more of similar output.
    and finally:

    Code:
    "VM Thread" prio=10 tid=0x080a5000 nid=0x15e2 runnable 

"VM Periodic Task Thread" prio=10 tid=0x080d5c00 nid=0x15f4 waiting on condition 

JNI global references: 1189

Heap
 def new generation   total 465472K, used 184121K [0x530f0000, 0x72a00000, 0x72a40000)
  eden space 413760K,  40% used [0x530f0000, 0x5d4f38f8, 0x6c500000)
  from space 51712K,  31% used [0x6c500000, 0x6d4cadb0, 0x6f780000)
  to   space 51712K,   0% used [0x6f780000, 0x6f780000, 0x72a00000)
 tenured generation   total 1034240K, used 26964K [0x72a40000, 0xb1c40000, 0xb1cf0000)
   the space 1034240K,   2% used [0x72a40000, 0x744950a8, 0x74495200, 0xb1c40000)
 compacting perm gen  total 24320K, used 24147K [0xb1cf0000, 0xb34b0000, 0xb5cf0000)
   the space 24320K,  99% used [0xb1cf0000, 0xb3484fe0, 0xb3485000, 0xb34b0000)
No shared spaces configured.

[tomcat launcher] got signal SIGTERM invoking stop
14-Apr-2008 13:03:04 org.apache.coyote.http11.Http11BaseProtocol pause
INFO: Pausing Coyote HTTP/1.1 on http-80
14-Apr-2008 13:03:04 org.apache.coyote.http11.Http11BaseProtocol pause
INFO: Pausing Coyote HTTP/1.1 on http-7071
14-Apr-2008 13:03:05 org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
WARNING: Cannot load DocumentFactory: null
14-Apr-2008 13:03:06 org.apache.coyote.http11.Http11BaseProtocol destroy
INFO: Stopping Coyote HTTP/1.1 on http-80
14-Apr-2008 13:03:06 org.apache.coyote.http11.Http11BaseProtocol destroy
INFO: Stopping Coyote HTTP/1.1 on http-7071
    Any ideas?
    Reply With Quote Reply With Quote
  2. 04-14-2008, 10:16 PM #2
    mmorse's Avatar
    mmorse
    mmorse is offline Moderator
    Send a message via ICQ to mmorse Send a message via AIM to mmorse Send a message via MSN to mmorse Send a message via Yahoo to mmorse Send a message via Skype™ to mmorse
    Join Date
    May 2006
    Location
    USA
    Posts
    6,241
    Rep Power
    20

    Default

    Off the top of my head no clue on that error, but wanted to confirm if that's 4.5.10_GA_1575.UBUNTU6? (from a prior post)
    http://www.zimbra.com/forums/announc...html#post62754
    -BTW 4.5.11 & 5.0.4 are out.
    -Mike Morse (MCode151)

    ZCS-to-ZCS Migrations & Moves | Admin Tools & Tidbits » ZimbraBlog.com | ZimbraCommunity.com
    Reply With Quote Reply With Quote
  3. 04-14-2008, 10:54 PM #3
    geoffDeGeoffGeoff
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    6

    Default

    zcs-4.5.10_GA_1575.UBUNTU6 is right.

    We are waiting for another server before we upgrade to the latest version.

    S
    Reply With Quote Reply With Quote
  4. 04-14-2008, 10:55 PM #4
    jholder's Avatar
    jholder
    jholder is offline Former Zimbran
    Send a message via Yahoo to jholder
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    19

    Default

    Need the whole log file.
    Reply With Quote Reply With Quote
  5. 04-15-2008, 07:35 PM #5
    geoffDeGeoffGeoff
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    6

    Default

    Attached is the catalina.out log for the day in question. Do you need any other logs?
    Attached Files Attached Files
    Reply With Quote Reply With Quote
  6. 04-15-2008, 08:09 PM #6
    geoffDeGeoffGeoff
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    6

    Default

    Our hosting company lost all connectivity with their dedicated servers yesterday :-(

    I spoke to them and at the exact time of our first outage last week they had some problem with the dns caching nameservers that may have led to problems with name resolution on our box.

    They have nothing logged at the time of the incident I describe above but it seems likely that it could have been the same problem.

    I attach the log from the time of the dns outage.

    Is there anything I can do to protect against this in the future or is it just an external problem that I cannot do anything about?

    Thanks,
    S
    Attached Files Attached Files
    Reply With Quote Reply With Quote
  7. 04-15-2008, 08:12 PM #7
    jholder's Avatar
    jholder
    jholder is offline Former Zimbran
    Send a message via Yahoo to jholder
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    19

    Default

    hmm
    Looks to me like you ran out of threads. How many IMAP clients do you have?

    Lots of locked threads:
    - locked <0x56921698> (a java.util.TaskQueue)
    - locked <0x6c5052d8> (a java.util.TaskQueue)
    - locked <0x737a90c0> (a EDU.oswego.cs.dl.util.concurrent.ClockDaemon)
    - locked <0x735e1730> (a java.util.TaskQueue)
    - locked <0x73032820> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x72c15c18> (a java.net.SocksSocketImpl)
    - locked <0x73032340> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730320d0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73031e60> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73031bf0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73031980> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73031710> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730314a0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73031230> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030fc0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030d50> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030ae0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030870> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030478> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73030208> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ff98> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302fd28> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302fab8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302f848> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302f5d8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302f368> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302f0f8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ee88> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ec18> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302e9a8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302e738> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302e4c8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302e258> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302dfe8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302dd78> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302db08> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302d898> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302d628> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302d3b8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302d148> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ced8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302cc68> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302c9f8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302c788> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302c518> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302be98> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302bc28> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302b9b8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302b748> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302b4d8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302b268> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302aff8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ad88> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7302ab18> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301cd98> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301cb28> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301c8b8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301c648> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301c3d8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301c168> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301bef8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x72b4d108> (a java.net.SocksSocketImpl)
    - locked <0x7301ba18> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301b7a8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301b538> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301b2c8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301b058> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301ade8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301ab78> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301a908> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301a698> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301a428> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7301a1b8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73019f48> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73019cd8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73019a68> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730197f8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73019588> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73019318> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730190a8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73018e38> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73018bc8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73018650> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730183e0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73018170> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73017f00> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73017c90> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73017a20> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730177b0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73017540> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730172d0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73017060> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73016df0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73016b80> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73016910> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730166a0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73016430> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730161c0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015f50> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015ce0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015a70> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015800> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015590> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73015320> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730150b0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73014e40> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73014bd0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73014960> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730146f0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73014480> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73014210> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013fa0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013d30> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013ac0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013850> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730135e0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013370> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73013100> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73012e90> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73012c20> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730129b0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73012740> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730124d0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73012260> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73011ff0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73011d80> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73011b10> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730118a0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73011630> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730113c0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73011150> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010ee0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010c70> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010a00> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010790> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010520> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x730102b0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x73010040> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300fdd0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300fb60> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300f8f0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300f680> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300f410> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300f1a0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300ef30> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300ecc8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300ea60> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300e7f8> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300e590> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300e328> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300e0c0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300de58> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300dbf0> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7300d988> (a org.apache.tomcat.util.threads.ThreadPool$ControlR unnable)
    - locked <0x7304dc08> (a com.zimbra.cs.store.FileBlobStore$IncomingDirector ySweeper)
    - locked <0x72ed7d68> (a java.net.SocksSocketImpl)
    - locked <0x72ed7be0> (a java.net.SocksSocketImpl)
    - locked <0x72ed7a68> (a java.net.SocksSocketImpl)
    - locked <0x72ed5ed0> (a java.net.SocksSocketImpl)
    - locked <0x72fa71a0> (a java.net.SocksSocketImpl)
    - locked <0x72f98338> (a sun.nio.ch.Util$1)
    - locked <0x72f98328> (a java.util.Collections$UnmodifiableSet)
    - locked <0x72f97fe0> (a sun.nio.ch.EPollSelectorImpl)
    - locked <0x72f93250> (a com.zimbra.cs.index.MailboxIndex$IndexWritersSweep er)
    - locked <0x72f22f40> (a java.util.TaskQueue)
    - locked <0x72f0d430> (a java.util.TaskQueue)
    - locked <0x72e3a3c8> (a com.sun.jndi.ldap.pool.PoolCleaner)
    - locked <0x72a40a80> (a java.lang.ref.ReferenceQueue$Lock)
    - locked <0x72a40980> (a java.lang.ref.Reference$Lock)
    Can you take a look at your mysql log and your mailbox.log and look for problems. Maybe a condition happened that caused it to run out of worker threads.
    Reply With Quote Reply With Quote
  8. 04-15-2008, 08:15 PM #8
    jholder's Avatar
    jholder
    jholder is offline Former Zimbran
    Send a message via Yahoo to jholder
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    19

    Default

    Quote Originally Posted by geoffDeGeoffGeoff View Post
    Is there anything I can do to protect against this in the future or is it just an external problem that I cannot do anything about?
    Yeah, i've actually heard of dns failures causing issues. One way to get around it is to run DNS in house, and have it point to your hosting providers as a external resolver. That way if it goes down, your internal will keep working.
    Reply With Quote Reply With Quote
  9. 04-15-2008, 08:24 PM #9
    geoffDeGeoffGeoff
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    6

    Default

    So do you think that running out of threads was related to the dns problem?

    By 'IMAP client' do you mean the number of people using a mail client to connect to our server using IMAP? If so then there cannot be more than a handful, most are using the web client and we have less than a hundred accounts that are actually in use.
    Reply With Quote Reply With Quote
  10. 04-15-2008, 08:40 PM #10
    geoffDeGeoffGeoff
    geoffDeGeoffGeoff is offline Active Member
    Join Date
    Dec 2007
    Location
    Chiang Mai, Thailand
    Posts
    29
    Rep Power
    6

    Default

    Here is an extract from mailbox.log at the time of the dns outage:
    Code:
    2008-04-11 00:00:19,107 WARN  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefOpenMailInNewWindow
2008-04-11 00:00:19,107 WARN  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefSentLifetime
2008-04-11 00:00:19,107 WARN  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefTrashLifetime
2008-04-11 00:00:19,107 WARN  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefJunkLifetime
2008-04-11 00:00:19,107 WARN  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefInboxUnreadLifetime
2008-04-11 00:00:19,110 INFO  [http-80-Processor84] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] SoapEngine - handler exception
com.zimbra.cs.account.AccountServiceException: invalid attr name: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]
	at com.zimbra.cs.account.AccountServiceException.INVALID_ATTR_NAME(AccountServiceException.java:105)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:289)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:268)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:248)
	at com.zimbra.cs.service.account.ModifyPrefs.handle(ModifyPrefs.java:60)
	at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:266)
	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:163)
	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:85)
	at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:220)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
	at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:152)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667)
	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
	at java.lang.Thread.run(Thread.java:619)
Caused by: javax.naming.directory.InvalidAttributeIdentifierException: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]; remaining name 'uid=iainkb,ou=people,dc=cmhmail,dc=com'
	at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3074)
	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2951)
	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2758)
	at com.sun.jndi.ldap.LdapCtx.c_modifyAttributes(LdapCtx.java:1441)
	at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_modifyAttributes(ComponentDirContext.java:255)
	at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.modifyAttributes(PartialCompositeDirContext.java:172)
	at javax.naming.directory.InitialDirContext.modifyAttributes(InitialDirContext.java:153)
	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttributes(LdapUtil.java:1052)
	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttrs(LdapUtil.java:590)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:286)
	... 25 more
2008-04-11 00:01:00,892 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefOpenMailInNewWindow
2008-04-11 00:01:00,893 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefSentLifetime
2008-04-11 00:01:00,893 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefTrashLifetime
2008-04-11 00:01:00,893 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefJunkLifetime
2008-04-11 00:01:00,893 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefInboxUnreadLifetime
2008-04-11 00:01:00,895 INFO  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] SoapEngine - handler exception
com.zimbra.cs.account.AccountServiceException: invalid attr name: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]
	at com.zimbra.cs.account.AccountServiceException.INVALID_ATTR_NAME(AccountServiceException.java:105)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:289)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:268)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:248)
	at com.zimbra.cs.service.account.ModifyPrefs.handle(ModifyPrefs.java:60)
	at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:266)
	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:163)
	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:85)
	at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:220)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
	at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:152)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667)
	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
	at java.lang.Thread.run(Thread.java:619)
Caused by: javax.naming.directory.InvalidAttributeIdentifierException: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]; remaining name 'uid=iainkb,ou=people,dc=cmhmail,dc=com'
	at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3074)
	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2951)
	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2758)
	at com.sun.jndi.ldap.LdapCtx.c_modifyAttributes(LdapCtx.java:1441)
	at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_modifyAttributes(ComponentDirContext.java:255)
	at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.modifyAttributes(PartialCompositeDirContext.java:172)
	at javax.naming.directory.InitialDirContext.modifyAttributes(InitialDirContext.java:153)
	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttributes(LdapUtil.java:1052)
	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttrs(LdapUtil.java:590)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:286)
	... 25 more
2008-04-11 00:01:41,916 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefOpenMailInNewWindow
2008-04-11 00:01:41,916 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefSentLifetime
2008-04-11 00:01:41,916 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefTrashLifetime
2008-04-11 00:01:41,916 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefJunkLifetime
2008-04-11 00:01:41,916 WARN  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] misc - checkValue: no attribute info for: zimbraPrefInboxUnreadLifetime
2008-04-11 00:01:41,919 INFO  [http-80-Processor79] [name=iainkb@cmhmail.com;ip=86.142.220.137;ua=zclient/5.0;] SoapEngine - handler exception
com.zimbra.cs.account.AccountServiceException: invalid attr name: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]
	at com.zimbra.cs.account.AccountServiceException.INVALID_ATTR_NAME(AccountServiceException.java:105)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:289)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:268)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrs(LdapProvisioning.java:248)
	at com.zimbra.cs.service.account.ModifyPrefs.handle(ModifyPrefs.java:60)
	at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:266)
	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:163)
	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:85)
	at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:220)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
	at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:152)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667)
	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
	at java.lang.Thread.run(Thread.java:619)
Caused by: javax.naming.directory.InvalidAttributeIdentifierException: [LDAP: error code 17 - zimbraPrefOpenMailInNewWindow: attribute type undefined]; remaining name 'uid=iainkb,ou=people,dc=cmhmail,dc=com'
	at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3074)
	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2951)
	at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2758)
	at com.sun.jndi.ldap.LdapCtx.c_modifyAttributes(LdapCtx.java:1441)
	at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_modifyAttributes(ComponentDirContext.java:255)
	at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.modifyAttributes(PartialCompositeDirContext.java:172)
	at javax.naming.directory.InitialDirContext.modifyAttributes(InitialDirContext.java:153)
	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttributes(LdapUtil.java:1052)
	at com.zimbra.cs.account.ldap.LdapUtil.modifyAttrs(LdapUtil.java:590)
	at com.zimbra.cs.account.ldap.LdapProvisioning.modifyAttrsInternal(LdapProvisioning.java:286)
	... 25 more
    Not really my area but this looks like some thread guzzling as a result of being unable to do something for one client. Probably the dns problem?
    Reply With Quote Reply With Quote
« Previous Thread | Next Thread »

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Zimbra fails after working for 2 weeks
    By Linsys in forum Administrators
    Replies: 10
    Last Post: 10-07-2008, 12:42 AM
  2. POP3 server dies every hour
    By jdossey7 in forum Administrators
    Replies: 12
    Last Post: 09-27-2007, 01:14 PM
  3. Clean Shutdown of Zimbra
    By swu in forum Administrators
    Replies: 1
    Last Post: 07-14-2007, 05:28 AM
  4. mystery solved: unclean shutdown causes redo.log to be owned by root:root
    By anand in forum Administrators
    Replies: 29
    Last Post: 03-06-2007, 12:14 PM
  5. Unclean Shutdown, root permissions, suspended queue and Debian
    By lee in forum Administrators
    Replies: 0
    Last Post: 09-23-2006, 10:25 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Forum Rules