Tomcat not starting after upgrade

[scottp]

After upgrading to 4.5.10 on Debian box Tomcat no longer starts.

Here is the catalina.out file:

7/12/2007 20:36:52 org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-80
7/12/2007 20:36:53 org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-443
7/12/2007 20:36:53 org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-7071
7/12/2007 20:36:53 org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 1396 ms
7/12/2007 20:36:53 org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
7/12/2007 20:36:53 org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.15
7/12/2007 20:36:53 org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
log4j:WARN No appenders could be found for logger (org.apache.catalina.startup.TldConfig).
log4j:WARN Please initialize the log4j system properly.
7/12/2007 20:36:54 org.apache.catalina.startup.ContextConfig applicationWebConfig
INFO: Missing application web.xml, using defaults only StandardEngine[Catalina].StandardHost[localhost].StandardContext[
/host-manager]
7/12/2007 20:36:54 org.apache.catalina.startup.ContextConfig applicationWebConfig
INFO: Missing application web.xml, using defaults only StandardEngine[Catalina].StandardHost[localhost].StandardContext[
/manager]
7/12/2007 20:36:54 org.apache.catalina.startup.HostConfig deployDescriptor
WARNING: A docBase /opt/zimbra/apache-tomcat-5.5.15/webapps/zimbra inside the host appBase has been specified, and will 
be ignored
log4j:WARN No appenders could be found for logger (org.apache.catalina.session.ManagerBase).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN No appenders could be found for logger (org.apache.catalina.session.ManagerBase).
log4j:WARN Please initialize the log4j system properly.
Zimbra server reserving server socket port=110 bindaddr=null ssl=false
Zimbra server reserving server socket port=995 bindaddr=null ssl=true
Zimbra server reserving server socket port=143 bindaddr=null ssl=false
Zimbra server reserving server socket port=993 bindaddr=null ssl=true
Zimbra server process is running as root, changing to user=zimbra uid=1000 gid=104
Zimbra server process, after change, is running with uid=1000 euid=1000 gid=104 egid=104
7/12/2007 20:36:59 org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-80
7/12/2007 20:37:00 org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-443
7/12/2007 20:37:00 org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-7071
7/12/2007 20:37:00 org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 1673 ms
7/12/2007 20:37:00 org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
7/12/2007 20:37:00 org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.15
7/12/2007 20:37:00 org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
log4j:WARN No appenders could be found for logger (org.apache.catalina.startup.TldConfig).
log4j:WARN Please initialize the log4j system properly.
7/12/2007 20:37:02 org.apache.catalina.startup.ContextConfig applicationWebConfig
INFO: Missing application web.xml, using defaults only StandardEngine[Catalina].StandardHost[localhost].StandardContext[
/host-manager]
7/12/2007 20:37:02 org.apache.catalina.startup.ContextConfig applicationWebConfig
INFO: Missing application web.xml, using defaults only StandardEngine[Catalina].StandardHost[localhost].StandardContext[
/manager]
7/12/2007 20:37:02 org.apache.catalina.startup.HostConfig deployDescriptor
WARNING: A docBase /opt/zimbra/apache-tomcat-5.5.15/webapps/zimbra inside the host appBase has been specified, and will 
be ignored
log4j:WARN No appenders could be found for logger (org.apache.catalina.session.ManagerBase).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN No appenders could be found for logger (org.apache.catalina.session.ManagerBase).
log4j:WARN Please initialize the log4j system properly.
Zimbra server reserving server socket port=110 bindaddr=null ssl=false
Zimbra server reserving server socket port=995 bindaddr=null ssl=true
Zimbra server reserving server socket port=143 bindaddr=null ssl=false
Zimbra server reserving server socket port=993 bindaddr=null ssl=true
Zimbra server process is running as root, changing to user=zimbra uid=1000 gid=104
Zimbra server process, after change, is running with uid=1000 euid=1000 gid=104 egid=104

The particularly odd messages above are on the web.xml lines. Comparing this with the backed up version I had, the application files in server/webapps/host-manager and server/webapps/manager are missing and were there before (prior versions).

This may be a red herring but I can't figure out any other reason it is failing.

Also, watching the processes, it starts correctly, starts using memory and then finally exits.

Any ideas?

This is kind of urgent as all my system is down and it is very hard to roll back because of schema changes.

Thanks

Scott

[phoenix]

Is that all that's in catalina.out? Is there anything in the other logs to indicate why it's failing? The error that you've posted in catalina.out is, as you've described, a red herring. It is just a warning about a missing folder and can be ignored.

[scottp]

Yes that is the whole of the catalina.out file.

Sometimes too I see a stacktrace file, but it is always blank.

Scott

[scottp]

Entries in zimbra.log for tomcat

Dec  7 23:04:09 zimbra zmtomcatmgr[12565]: manager started tomcat/JVM with pid 12566
Dec  7 23:04:18 zimbra zmtomcatmgr[12565]: manager woke up from wait on tomcat/JVM with pid 12566
Dec  7 23:04:18 zimbra zmtomcatmgr[12565]: manager started tomcat/JVM with pid 12683
Dec  7 23:04:27 zimbra zmtomcatmgr[12565]: manager woke up from wait on tomcat/JVM with pid 12683
Dec  7 23:04:27 zimbra zmtomcatmgr[12565]: tomcat/JVM exited twice in 9 seconds (tolerance=60)
Dec  7 23:05:09 zimbra zmtomcatmgr[13588]: status requested
Dec  7 23:05:09 zimbra zmtomcatmgr[13588]: stale pid 12565 found in /opt/zimbra/log/zmtomcatmgr.pid: No such process
Dec  7 23:05:09 zimbra zmtomcatmgr[13588]: assuming no other instance is running
Dec  7 23:05:09 zimbra zmtomcatmgr[13588]: no manager process is running
Dec  7 23:06:04 zimbra zmtomcatmgr[14472]: status requested
Dec  7 23:06:04 zimbra zmtomcatmgr[14472]: stale pid 12565 found in /opt/zimbra/log/zmtomcatmgr.pid: No such process
Dec  7 23:06:04 zimbra zmtomcatmgr[14472]: assuming no other instance is running
Dec  7 23:06:04 zimbra zmtomcatmgr[14472]: no manager process is running
Dec  7 23:08:04 zimbra zmtomcatmgr[15998]: status requested
Dec  7 23:08:04 zimbra zmtomcatmgr[15998]: stale pid 12565 found in /opt/zimbra/log/zmtomcatmgr.pid: No such process
Dec  7 23:08:04 zimbra zmtomcatmgr[15998]: assuming no other instance is running
Dec  7 23:08:04 zimbra zmtomcatmgr[15998]: no manager process is running
Dec  7 23:10:07 zimbra zmtomcatmgr[18121]: status requested
Dec  7 23:10:07 zimbra zmtomcatmgr[18121]: stale pid 12565 found in /opt/zimbra/log/zmtomcatmgr.pid: No such process
Dec  7 23:10:07 zimbra zmtomcatmgr[18121]: assuming no other instance is running
Dec  7 23:10:07 zimbra zmtomcatmgr[18121]: no manager process is running
Dec  7 23:11:15 zimbra zmtomcatmgr[25004]: threaddump requested
Dec  7 23:11:15 zimbra zmtomcatmgr[25004]: stale pid 12565 found in /opt/zimbra/log/zmtomcatmgr.pid: No such process
Dec  7 23:11:15 zimbra zmtomcatmgr[25004]: assuming no other instance is running
Dec  7 23:11:15 zimbra zmtomcatmgr[25004]: no manager process is running
Dec  7 23:11:17 zimbra zmtomcatmgr[25011]: stop requested
Dec  7 23:11:17 zimbra zmtomcatmgr[25011]: stale pid 12565 found in /opt/zimbra/log/zmtomcatmgr.pid: No such process
Dec  7 23:11:17 zimbra zmtomcatmgr[25011]: assuming no other instance is running
Dec  7 23:11:17 zimbra zmtomcatmgr[25011]: no manager process is running
Dec  7 23:11:22 zimbra zmtomcatmgr[25013]: status requested
Dec  7 23:11:22 zimbra zmtomcatmgr[25013]: stale pid 12565 found in /opt/zimbra/log/zmtomcatmgr.pid: No such process
Dec  7 23:11:22 zimbra zmtomcatmgr[25013]: assuming no other instance is running
Dec  7 23:11:22 zimbra zmtomcatmgr[25013]: no manager process is running
Dec  7 23:11:22 zimbra zmtomcatmgr[25014]: status requested
Dec  7 23:11:22 zimbra zmtomcatmgr[25014]: stale pid 12565 found in /opt/zimbra/log/zmtomcatmgr.pid: No such process
Dec  7 23:11:22 zimbra zmtomcatmgr[25014]: assuming no other instance is running
Dec  7 23:11:22 zimbra zmtomcatmgr[25014]: no manager process is running

And finally got a stacktrace file - here is the important bit, but it is also attached:

Heap
 def new generation   total 464832K, used 78021K [0x532b0000, 0x72b10000, 0x72b50000)
  eden space 413184K,  18% used [0x532b0000, 0x57ee1430, 0x6c630000)
  from space 51648K,   0% used [0x6c630000, 0x6c630000, 0x6f8a0000)
  to   space 51648K,   0% used [0x6f8a0000, 0x6f8a0000, 0x72b10000)
 tenured generation   total 1032896K, used 5830K [0x72b50000, 0xb1c00000, 0xb1cb0000)
   the space 1032896K,   0% used [0x72b50000, 0x73101aa8, 0x73101c00, 0xb1c00000)
 compacting perm gen  total 19968K, used 19874K [0xb1cb0000, 0xb3030000, 0xb5cb0000)
   the space 19968K,  99% used [0xb1cb0000, 0xb30189f8, 0xb3018a00, 0xb3030000)
No shared spaces configured.

Scott

[scottp]

Here is the stacktrace