LinkBack URL
About LinkBacks
[SOLVED] zmmailboxdctl is not running; unable to reserve port 7025
Hi list, my first post here, but I'm at wit's end. Basically, the cleaners pulled the plug on a Zimbra Release 5.0.13_GA_2791.UBUNTU8 UBUNTU8 installation at a client's office, while I was on vacation. The machine worked fine for over a year. While I was gone, the client had someone else look at it. Not sure exactly what was done, unfortunately.
When I got back, I restored from a backup several days before the crash, but it still doesn't work. The other person had played with the hosts file and DNS, and unfortunately, I can't remember how it was originally set up. The server is behind a firewall, and is the public DNS server for the domain, so its hostname and internal IP address don't match, but I think I have that covered.
It appears that port 7025 never gets created, so mailboxd doesn't work.
First, here's what everybody asks for:
Check hostname resolution:
I set up an IP alias on eth0:0 so the local host could resolve mail.tkdb.ca to itself:Code:root@mail:/var/log# cat /etc/hosts 127.0.0.1 localhost.localdomain localhost 192.168.0.199 mail.tkdb.ca mail # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts root@mail:/var/log# cat /etc/resolv.conf # Generated by NetworkManager search tkdb.ca #nameserver 192.168.0.199 nameserver 127.0.0.1 root@mail:~# host `hostname` mail.tkdb.ca has address 70.77.224.231 root@mail:~# dig tkdb.ca mx ; <<>> DiG 9.4.2-P2 <<>> tkdb.ca mx ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64748 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 2 ;; QUESTION SECTION: ;tkdb.ca. IN MX ;; ANSWER SECTION: tkdb.ca. 3600 IN MX 10 mail.tkdb.ca. ;; AUTHORITY SECTION: tkdb.ca. 3600 IN NS ns.tkdb.ca. ;; ADDITIONAL SECTION: mail.tkdb.ca. 3600 IN A 70.77.224.231 ns.tkdb.ca. 3600 IN A 70.77.224.231 ;; Query time: 0 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Thu Jul 1 08:52:31 2010 ;; MSG SIZE rcvd: 95 root@mail:~# dig tkdb.ca any ; <<>> DiG 9.4.2-P2 <<>> tkdb.ca any ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1306 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 2 ;; QUESTION SECTION: ;tkdb.ca. IN ANY ;; ANSWER SECTION: tkdb.ca. 3600 IN A 70.77.224.231 tkdb.ca. 3600 IN SOA ns.tkdb.ca. hostmaster.tkdb.ca. 2009031803 3600 3600 604800 604800 tkdb.ca. 3600 IN NS ns.tkdb.ca. tkdb.ca. 3600 IN MX 10 mail.tkdb.ca. ;; ADDITIONAL SECTION: ns.tkdb.ca. 3600 IN A 70.77.224.231 mail.tkdb.ca. 3600 IN A 70.77.224.231 ;; Query time: 0 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Thu Jul 1 08:53:05 2010 ;; MSG SIZE rcvd: 158 root@mail:~#
OK, that seems to work, now BEFORE firing up Zimbra, check the local ports:Code:root@mail:/var/log# ifconfig eth0 Link encap:Ethernet HWaddr 00:15:17:26:c6:9c inet addr:192.168.0.199 Bcast:192.168.0.255 Mask:255.255.255.0 inet6 addr: fe80::215:17ff:fe26:c69c/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:64062 errors:0 dropped:0 overruns:0 frame:0 TX packets:50705 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:51786501 (49.3 MB) TX bytes:5154874 (4.9 MB) Base address:0x1000 Memory:e1c20000-e1c40000 eth0:0 Link encap:Ethernet HWaddr 00:15:17:26:c6:9c inet addr:70.77.224.231 Bcast:70.255.255.255 Mask:255.255.255.255 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Base address:0x1000 Memory:e1c20000-e1c40000 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:336140 errors:0 dropped:0 overruns:0 frame:0 TX packets:336140 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:199374882 (190.1 MB) TX bytes:199374882 (190.1 MB) root@mail:/var/log# ping mail.tkdb.ca PING mail.tkdb.ca (192.168.0.199) 56(84) bytes of data. 64 bytes from mail.tkdb.ca (192.168.0.199): icmp_seq=1 ttl=64 time=0.014 ms 64 bytes from mail.tkdb.ca (192.168.0.199): icmp_seq=2 ttl=64 time=0.006 ms 64 bytes from mail.tkdb.ca (192.168.0.199): icmp_seq=3 ttl=64 time=0.005 ms --- mail.tkdb.ca ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.005/0.008/0.014/0.004 ms root@mail:/var/log#
...nope, nothing there on the SMTP and 7025 ports...Code:root@mail:~# telnet mail.tkdb.ca 7025 Trying 192.168.0.199... telnet: Unable to connect to remote host: Connection refused root@mail:~# telnet mail.tkdb.ca 25 Trying 192.168.0.199... telnet: Unable to connect to remote host: Connection refused
...DNS works tho, so we can connect to ports...Code:root@mail:~# telnet mail.tkdb.ca 53 Trying 192.168.0.199... Connected to mail.tkdb.ca. Escape character is '^]'. ^] telnet> quit Connection closed.
Now, fire up Zimbra (as zimbra user):
OK, so it looks like the SMTP port is being created fine, but not 7025. Verify that:Code:zimbra@mail:~$ zmcontrol start Host mail.tkdb.ca Starting logger...Done. Starting mailbox...Done. Starting antispam...Done. Starting snmp...Done. Starting mta...Done. Starting stats...Done. zimbra@mail:~$ zmcontrol status Host mail.tkdb.ca antispam Running ldap Running logger Running mailbox Stopped zmmailboxdctl is not running. mta Running snmp Running stats Running zimbra@mail:~$ cd /opt/zimbra/log zimbra@mail:~/log$ tail -80 zmmailboxd.out | more : lotsa stuff here : 842 INFO [Shutdown] log - Shutdown hook executing 842 INFO [Shutdown] log - Shutdown hook complete CompilerOracle: exclude com/zimbra/cs/session/SessionMap putAndPrune CompilerOracle: exclude com/zimbra/cs/mailbox/MailItem delete 0 INFO [main] log - Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog Zimbra server reserving server socket port=143 bindaddr=null ssl=false Zimbra server reserving server socket port=993 bindaddr=null ssl=true Zimbra server reserving server socket port=7025 bindaddr=null ssl=false 1701 WARN [main] log - Config error at <Call name="open"/> 1701 WARN [main] log - Config error at <Ref id="http"><Call name="open"/></Ref> 1702 WARN [main] log - EXCEPTION java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.mortbay.xml.XmlConfiguration.call(XmlConfiguration.java:534) at org.mortbay.xml.XmlConfiguration.configure(XmlConfiguration.java:244) at org.mortbay.xml.XmlConfiguration.refObj(XmlConfiguration.java:638) at org.mortbay.xml.XmlConfiguration.configure(XmlConfiguration.java:252) at org.mortbay.xml.XmlConfiguration.configure(XmlConfiguration.java:206) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:968) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.mortbay.start.Main.invokeMain(Main.java:183) at org.mortbay.start.Main.start(Main.java:497) at org.mortbay.start.Main.main(Main.java:115) Caused by: java.net.BindException: Address already in use at sun.nio.ch.Net.bind(Native Method) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:119) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59) at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:211) ... 17 more 1705 WARN [main] log - Nested in java.lang.reflect.InvocationTargetException: java.net.BindException: Address already in use at sun.nio.ch.Net.bind(Native Method) : lots more stuff here :
...yup, 25 worksCode:root@mail:~# telnet mail.tkdb.ca 25 Trying 192.168.0.199... Connected to mail.tkdb.ca. Escape character is '^]'. 220 mail.tkdb.ca ESMTP Postfix ^] telnet> quit Connection closed.
...but 7025 still doesn't! AaarrgghhhCode:root@mail:~# telnet mail.tkdb.ca 7025 Trying 192.168.0.199... telnet: Unable to connect to remote host: Connection refused root@mail:~#
Here's /var/log/zimbra.log for this session:
Lots of the latter 2 types of messages (actual email address changed to protect the innocent!)Code:: : Jul 1 09:04:37 mail zmmailboxdmgr[1379]: status requested Jul 1 09:04:37 mail zmmailboxdmgr[1379]: stale pid 28908 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process Jul 1 09:04:37 mail zmmailboxdmgr[1379]: assuming no other instance is running Jul 1 09:04:37 mail zmmailboxdmgr[1379]: file /opt/zimbra/log/zmmailboxd.pid does not exist Jul 1 09:04:37 mail zmmailboxdmgr[1379]: assuming no other instance is running Jul 1 09:04:37 mail zmmailboxdmgr[1379]: no manager process is running Jul 1 09:04:37 mail zmmailboxdmgr[1386]: start requested Jul 1 09:04:37 mail zmmailboxdmgr[1386]: checking if another instance of manager is already running Jul 1 09:04:37 mail zmmailboxdmgr[1386]: stale pid 28908 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process Jul 1 09:04:37 mail zmmailboxdmgr[1386]: assuming no other instance is running Jul 1 09:04:37 mail zmmailboxdmgr[1386]: file /opt/zimbra/log/zmmailboxd.pid does not exist Jul 1 09:04:37 mail zmmailboxdmgr[1386]: assuming no other instance is running Jul 1 09:04:37 mail zmmailboxdmgr[1387]: wrote manager pid 1387 to /opt/zimbra/log/zmmailboxd_manager.pid Jul 1 09:04:37 mail zmmailboxdmgr[1388]: wrote java pid 1388 to /opt/zimbra/log/zmmailboxd_java.pid : : Jul 1 09:04:37 mail zmmailboxdmgr[1379]: status requested Jul 1 09:04:37 mail zmmailboxdmgr[1379]: stale pid 28908 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process Jul 1 09:04:37 mail zmmailboxdmgr[1379]: assuming no other instance is running Jul 1 09:04:37 mail zmmailboxdmgr[1379]: file /opt/zimbra/log/zmmailboxd.pid does not exist Jul 1 09:04:37 mail zmmailboxdmgr[1379]: assuming no other instance is running Jul 1 09:04:37 mail zmmailboxdmgr[1379]: no manager process is running Jul 1 09:04:37 mail zmmailboxdmgr[1386]: start requested Jul 1 09:04:37 mail zmmailboxdmgr[1386]: checking if another instance of manager is already running Jul 1 09:04:37 mail zmmailboxdmgr[1386]: stale pid 28908 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process Jul 1 09:04:37 mail zmmailboxdmgr[1386]: assuming no other instance is running Jul 1 09:04:37 mail zmmailboxdmgr[1386]: file /opt/zimbra/log/zmmailboxd.pid does not exist Jul 1 09:04:37 mail zmmailboxdmgr[1386]: assuming no other instance is running Jul 1 09:04:37 mail zmmailboxdmgr[1387]: wrote manager pid 1387 to /opt/zimbra/log/zmmailboxd_manager.pid Jul 1 09:04:37 mail zmmailboxdmgr[1388]: wrote java pid 1388 to /opt/zimbra/log/zmmailboxd_java.pid Jul 1 09:04:37 mail zimbramon[32332]: 32332:info: Starting antispam via zmcontrol Jul 1 09:04:37 mail zmmailboxdmgr[1387]: manager started mailboxd/JVM with pid 1388 : : Jul 1 09:04:39 mail zmmailboxdmgr[1387]: manager woke up from wait on mailboxd/JVM with pid 1388 Jul 1 09:04:39 mail zmmailboxdmgr[1387]: manager started mailboxd/JVM with pid 1544 Jul 1 09:04:39 mail zmmailboxdmgr[1544]: wrote java pid 1544 to /opt/zimbra/log/zmmailboxd_java.pid Jul 1 09:04:41 mail zmmailboxdmgr[1387]: manager woke up from wait on mailboxd/JVM with pid 1544 Jul 1 09:04:41 mail zmmailboxdmgr[1387]: mailboxd/JVM exited twice in 2 seconds (tolerance=60) : : Jul 1 09:04:49 mail postfix/lmtp[1888]: connect to mail.tkdb.ca[70.77.224.231]: Connection refused (port 7025) Jul 1 09:04:49 mail postfix/lmtp[1887]: connect to mail.tkdb.ca[70.77.224.231]: Connection refused (port 7025) : : Jul 1 09:04:50 mail postfix/lmtp[1888]: 15C76DBC0ED: to=<somebody @ tkdb.ca>, relay=none, delay=44962, delays=44962/0.03/0/0, dsn=4.4.1, status=deferred (connect to mail.tkdb.ca[70.77.224.231]: Connection refused) Jul 1 09:04:50 mail postfix/lmtp[1906]: 17014DBC0F3: to=<somebody @ tkdb.ca>, relay=none, delay=44765, delays=44765/0.07/0/0, dsn=4.4.1, status=deferred (connect to mail.tkdb.ca[70.77.224.231]: Connection refused) : ...and so on...
There are also lots of these:
So, can someone help me with this? The clients have been without email for a week, so I'd really appreciate if somebody could get back to me as soon as possible... Thanks!Code:Jul 1 09:06:02 mail zmmailboxdmgr[2891]: status requested Jul 1 09:06:02 mail zmmailboxdmgr[2891]: stale pid 1387 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process Jul 1 09:06:02 mail zmmailboxdmgr[2891]: assuming no other instance is running Jul 1 09:06:02 mail zmmailboxdmgr[2891]: file /opt/zimbra/log/zmmailboxd.pid does not exist Jul 1 09:06:02 mail zmmailboxdmgr[2891]: assuming no other instance is running Jul 1 09:06:02 mail zmmailboxdmgr[2891]: no manager process is running
Originally Posted by TomHaws 
