zmamavisdctl and tomcat are not running (anymore)

[oerton]

Hello,

Trying to setup zimbra to manage several domains, I am affraid I have broken it.
To sum up :

1/ I have zimbra on my own server, which is used by several people and domains. It used to work quite well.

2/
I discovered that a few emails never reached my server. I think it was due to sender SMTPs, doing 'reverse dns' on mine, and rejecting it because there was localhost!=reverse dns. So I've modified my 'reverse dns', to have localhost=reverse dns. But doing so, I've had to modify my hostname.

3/
Since then, zimbra doesn't work any more...
After a lot of reboots, and 'zmcontrol stop/start', I now have :

zimbra@mydomain.com:~$ zmcontrol status
Host mydomain.com
antivirus Stopped
zmamavisdctl is not running
imapproxy Running
ldap Running
logger Running
mailbox Stopped
tomcat is not running
mta Running
snmp Running
spell Running


4/
I made a 'zmsetservername mydomain.com' with no success

5/
I made also a 'zmfixperms' to be sure rights are good

6/
a few errors in /opt/zimbra/tomcat/logs are :

Jun 5, 2007 6:41:32 PM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-90
Jun 5, 2007 6:41:32 PM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-7071
Jun 5, 2007 6:41:32 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 997 ms
Jun 5, 2007 6:41:32 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Jun 5, 2007 6:41:32 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.15
Jun 5, 2007 6:41:32 PM org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
log4j:WARN No appenders could be found for logger (org.apache.catalina.startup.TldConfig).
log4j:WARN Please initialize the log4j system properly.
Jun 5, 2007 6:41:33 PM org.apache.catalina.core.StandardContext resourcesStart
SEVERE: Error starting static Resources
java.lang.IllegalArgumentException: Document base /home/opt/zimbra/apache-tomcat-5.5.15/server/webapps/host-manager does not exist or is not a readable directory
at org.apache.naming.resources.FileDirContext.setDocB ase(FileDirContext.java:140)
...
Jun 5, 2007 6:41:33 PM org.apache.catalina.core.StandardContext start
SEVERE: Error in resourceStart()
Jun 5, 2007 6:41:33 PM org.apache.catalina.core.StandardContext start
SEVERE: Error getConfigured
Jun 5, 2007 6:41:33 PM org.apache.catalina.core.StandardContext start
SEVERE: Context [/host-manager] startup failed due to previous errors
Jun 5, 2007 6:41:33 PM org.apache.catalina.core.StandardContext stop
INFO: Container org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager] has not been started
Jun 5, 2007 6:41:33 PM org.apache.catalina.core.StandardContext resourcesStart
SEVERE: Error starting static Resources
java.lang.IllegalArgumentException: Document base /home/opt/zimbra/apache-tomcat-5.5.15/server/webapps/manager does not exist or is not a readable directory
at org.apache.naming.resources.FileDirContext.setDocB ase(FileDirContext.java:140)
...
Jun 5, 2007 6:41:33 PM org.apache.catalina.core.StandardContext start
SEVERE: Error in resourceStart()
Jun 5, 2007 6:41:33 PM org.apache.catalina.core.StandardContext start
SEVERE: Error getConfigured
Jun 5, 2007 6:41:33 PM org.apache.catalina.core.StandardContext start
SEVERE: Context [/manager] startup failed due to previous errors
Jun 5, 2007 6:41:33 PM org.apache.catalina.core.StandardContext stop
INFO: Container org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager] has not been started
Jun 5, 2007 6:41:33 PM org.apache.catalina.startup.HostConfig deployDescriptor
WARNING: A docBase /opt/zimbra/apache-tomcat-5.5.15/webapps/zimbra inside the host appBase has been specified, and will be ignored
log4j:WARN No appenders could be found for logger (org.apache.catalina.session.ManagerBase).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN No appenders could be found for logger (org.apache.catalina.session.ManagerBase).
log4j:WARN Please initialize the log4j system properly.
Zimbra server reserving server socket port=110 bindaddr=null ssl=false
Fatal error: terminating: PrivilegedServlet init failed
java.net.BindException: Address already in use

do I have lost webapps directories ???

how is it possible the 110 port could be already in use ?

any help is welcome. I do not see what I could do now...
Is it possible to launch the zimbra install again to clean the server, without loosing any zimbra data ???

Thanks for your answers.

Olivier.

[mmorse]

2/ [...] But doing so, I've had to modify my hostname.

There is an actual process for changing your hostname -search the forums/wiki

You make a backup right?
If you don't have a backup to go to...let us know so we can press on with helping you.
Otherwise restore your backup and undo anything you did in your step 2 above.

[oerton]

Thanks for your answer.

I indeed have tried the following steps :
Changing Your Hostname

...but, no luck : unable to make the very first step... to much error messages.

I have nevertheless changed the domain into config files. but this doen's help more. and zmprov doesn't work either.

So... do you know what I could try before doing an uninstall/install ,without losing all data (as backup was only planned... ) ? no luck.

thanks.

[mmorse]

you posted duplicates 4 times and it made them moderated posts
-I approved one

[mmorse]

probably should have run this before your changed it

su - zimbra
zmcontrol stop
/opt/zimbra/libexec/zmsetservername <NEW_HOSTNAME>
Now update the ip address(es) and host name(s) for the server(s).

well start filling us in on the situation:
ie: make us a mental (or actual) diagram
all the hostnames etc of the current setup-and previous setup
host file contents etc
(edit/use alternate domain names & IP's if you want to protect your/companies identity of course)

[oerton]

Yes, it's what I did... and it broke everything.

To sum up the situation :

1/
I use a server, whose name was given at the beginning : s000001.ovh.net.
I have a admin console at ovh.net to set up the reverse DNS.

2/
I installed zimbra with no problem, and setted it up to have several domains :
domain1.com, domain2.com, domain3.com. I modified registrar DNS to have each of those domain point to s000001.ovh.net IP (A and MX). It works

3/
Some people (from big companies mainly) complain some emails they sent never reach some of my domain (people@domain1.com receive some emails, but all sent from same place are lost). I thought it was due to 'reverse DNS', as reverse DNS of domain1.com was s000001.ovh.net...

4/
To be sure, I modified 'reverse DNS' to set it up to domain1.com (very first step to understand)

5/
zimbra complains... so I made the 'zmsetservername ...'

6/
modified the hostname into /opt/zimbra/config.17436

7/
zmcontrol start/stop...

and it gives the log I join into the zip file.

Tell me if it helps, or if any information lacks.

Thanks.

Olivier.

[mmorse]

Your first severe messages seem to be looking in /home for tomcat manager:

SEVERE: Error starting static Resources
java.lang.IllegalArgumentException: Document base /home/opt/zimbra/apache-tomcat-5.5.15/server/webapps/host-manager does not exist or is not a readable directory

Any insight from your end?

One possible missing step...your /etc/hosts file?

From 2006-but the question is probably still valid-some zimbra engineer needs to reverify that that zmsetservername script is ok
server hostname change -very old/2006
Everyone seems to have no issues changing ip's (as it's not really that involved).
However, it would appear that the hostname changing method needs a quick re-inspection.
for instance more recent/last month:
Change hostname and IP Address!-failure

BUT another recent 2007 success:

I just changed my hostname on Zimbra 4.5.3 and RHEL 4.4AS.

1. Update DNS server
2. Edit /etc/sysconfig/network
3. su - zimbra -c /opt/zimbra/libexec/zmsetservername new.name.com
4. Reboot

The only "problem" I see is in the Admin UI, if you look at server status the new name is listed, but the old server name is also listed with a red X on everything except for the blue check marks next to LDAP and Logger. However, that all went away after a couple minutes. There must be a cronjob that updates a status log.

Of note I also found this interesting:

I decided to just try ZmSetServerName
If you follow the directions exactly (stopping the services first), you get an error. If, instead, you just run the command, there is no error, and it stops the services automatically. That seems to have solved my "problem", or at least alleviated my fears. Since the internal DNS lists both domains fully, I'm not too worried about any leftover domain issues.

oerton btw-If you didn't know, these are harmless:

Jun 6 09:30:23 domain1 slapd[9277]: <= bdb_equality_candidates: (zimbraDomainType) index_param failed (18)
Jun 6 09:30:23 domain1 slapd[9277]: <= bdb_equality_candidates: (zimbraMailStatus) index_param failed (18)
Jun 6 09:30:23 domain1 slapd[9277]: <= bdb_equality_candidates: (zimbraDomainType) index_param failed (18)
Jun 6 09:30:23 domain1 slapd[9277]: <= bdb_equality_candidates: (zimbraMailStatus) index_param failed (18)

[oerton]

Hello,

Dealing with tomcat, the path is working, as I have a link from /opt to /home/opt, because I had to few disk space on /opt to install zimbra.
It used to work with this linke.

I read the other post, with no more success.

I tried again a zmsetservername, zmfixperms, but I still have the same errors described before. same logs... and zimbra processes down.

hosts and DNS config are fine. no solution.

Could be the idea to reinstall zimbra (open source version or the network edition I plan to try) a good idea ? Is it possible to hope to find my data back ? Does a backup sound possible ?

Thanks again for your support.

[sixx]

Hello.

I'm experiencing exactly the same problem.
As I was trying to connect to the imap port with Outlook, the Admin interface suddenly crashed. Since then, whatever I try, in the end I get:

[zimbra@enola ~]$ zmcontrol status
Host some.host.tld
        antispam                Running
        antivirus               Running
        imapproxy               Running
        ldap                    Running
        logger                  Running
        mailbox                 Stopped
                tomcat is not running
        mta                     Running
        snmp                    Running
        spell                   Running

catalina.out says:

Jun 9, 2007 7:09:09 AM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-80
Jun 9, 2007 7:09:10 AM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-7071
Jun 9, 2007 7:09:10 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 1464 ms
Jun 9, 2007 7:09:10 AM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Jun 9, 2007 7:09:10 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.15
Jun 9, 2007 7:09:10 AM org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
log4j:WARN No appenders could be found for logger (org.apache.catalina.startup.TldConfig).
log4j:WARN Please initialize the log4j system properly.
Jun 9, 2007 7:09:11 AM org.apache.catalina.core.StandardContext resourcesStart
SEVERE: Error starting static Resources
java.lang.IllegalArgumentException: Document base /opt/zimbra/apache-tomcat-5.5.15/server/webapps/host-manager does not exist or is not a readable directory
        at org.apache.naming.resources.FileDirContext.setDocBase(FileDirContext.java:140)
        at org.apache.catalina.core.StandardContext.resourcesStart(StandardContext.java:3812)
        at org.apache.catalina.core.StandardContext.start(StandardContext.java:3983)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:759)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:739)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:524)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:603)
        at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:535)
        at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:470)
        at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1118)
        at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:310)
        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1020)
        at org.apache.catalina.core.StandardHost.start(StandardHost.java:718)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1012)
        at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:442)
        at org.apache.catalina.core.StandardService.start(StandardService.java:450)
        at org.apache.catalina.core.StandardServer.start(StandardServer.java:700)
        at org.apache.catalina.startup.Catalina.start(Catalina.java:551)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:275)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:406)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at com.zimbra.cs.launcher.TomcatLauncher.start(TomcatLauncher.java:52)
        at com.zimbra.cs.launcher.TomcatLauncher.main(TomcatLauncher.java:74)

Help ?

[sixx]

Forgot to mention that the perms on redolog are ok and that sometimes (randomly i think), I also get:

 antivirus Stopped
zmamavisdctl is not running