zmmailboxdctl is not running

[gargola]

Hi!

I have been having some trouble with the FOSS edition of Zimbra.

The server was running w/o problems during the last 2 months. We rebooted the server last night.

After a server reboot zimbra did not load up anymore, and the following where the results of my tests:

zimbra@poseidon:~$ zmcontrol status
Host wglatina.dyndns.org
antispam Running
antivirus Running
ldap Running
logger Running
mailbox Stopped
zmmailboxdctl is not running
mta Running
snmp Running
spell Running
stats Running


zimbra@poseidon:~$ zmprov
ERROR: zclient.IO_ERROR (invoke Connection refused, server: localhost) (cause: java.net.ConnectException Connection refused)
zimbra@poseidon:~$

zimbra@poseidon:~$ zmprov gs wglatina.dyndns.org
ERROR: zclient.IO_ERROR (invoke Connection refused, server: localhost) (cause: java.net.ConnectException Connection refused)
zimbra@poseidon:~$

tail -f /var/log/zimbra.log
Mar 25 12:40:14 poseidon zimbramon[22137]: 22137:info: 2008-03-25 12:40:01, STATUS: wglatina.dyndns.org: antivirus: Running
Mar 25 12:40:14 poseidon zimbramon[22137]: 22137:info: 2008-03-25 12:40:01, STATUS: wglatina.dyndns.org: ldap: Running
Mar 25 12:40:14 poseidon zimbramon[22137]: 22137:info: 2008-03-25 12:40:01, STATUS: wglatina.dyndns.org: logger: Running
Mar 25 12:40:14 poseidon zimbramon[22137]: 22137:info: 2008-03-25 12:40:01, STATUS: wglatina.dyndns.org: mailbox: Stopped
Mar 25 12:40:14 poseidon zimbramon[22137]: 22137:info: 2008-03-25 12:40:01, STATUS: wglatina.dyndns.org: mta: Running
Mar 25 12:40:14 poseidon zimbramon[22137]: 22137:info: 2008-03-25 12:40:01, STATUS: wglatina.dyndns.org: snmp: Running
Mar 25 12:40:14 poseidon zimbramon[22137]: 22137:info: 2008-03-25 12:40:01, STATUS: wglatina.dyndns.org: spell: Running
Mar 25 12:40:14 poseidon zimbramon[22137]: 22137:info: 2008-03-25 12:40:01, STATUS: wglatina.dyndns.org: stats: Running


The following message is also displayed
Mar 25 12:17:10 poseidon zmmailboxdmgr[16321]: manager woke up from wait on mailboxd/JVM with pid 16365
Mar 25 12:17:10 poseidon zmmailboxdmgr[16321]: mailboxd/JVM exited twice in 3 seconds (tolerance=60)

I tryed to see what the problem is, however I have no clue. I also tryed to reinstall the zimbra software (running on ubuntu LTS) and did not help.

root@poseidon:~# hostname
poseidon.wglatina.dyndns.org
root@poseidon:~# hostname -f
poseidon.wglatina.dyndns.org
root@poseidon:~# dig mx wglatina.dyndns.org

; <<>> DiG 9.3.2 <<>> mx wglatina.dyndns.org
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39490
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 1, ADDITIONAL: 2

;; QUESTION SECTION:
;wglatina.dyndns.org. IN MX

;; ANSWER SECTION:
wglatina.dyndns.org. 604800 IN MX 20 poseidon.wglatina.dyndns.org.
wglatina.dyndns.org. 604800 IN MX 10 wglatina.dyndns.org.


Does anyone have any clue what might be happening?

[uxbod]

Welcome to the forums

What O/S and version of Zimbra ?

Please post /opt/zimbra/log/zmmailboxd.out

[gargola]

Hi,

OS: Ubuntu 6.06 LTS (Server)
Version: 5.0.2

[gargola]

see attachment

[uxbod]

Looks like something is already using the ports :-

Zimbra server reserving server socket port=143 bindaddr=null ssl=false\
Zimbra server reserving server socket port=993 bindaddr=null ssl=true\
Fatal error: exception while binding to ports\
java.net.BindException: Address already in use\

While Zimbra is down could you run the following :-

netstat -an

so we can see what ports are in use.

[gargola]

Yes I thought so too, however a netstat -an says does not lead into anything:

root@poseidon:~# netstat -an
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:993 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:49409 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:2049 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:139 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN
tcp 0 0 192.168.178.2:53 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:19000 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:953 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:19001 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:445 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:925 0.0.0.0:* LISTEN
tcp 0 0 192.168.178.2:39660 129.13.251.52:24 ESTABLISHED
tcp6 0 0 :::1100 :::* LISTEN
tcp6 0 0 :::2222 :::* LISTEN
tcp6 0 0 :::53 :::* LISTEN
tcp6 0 0 :::22 :::* LISTEN
tcp6 0 0 ::1:19000 :::* LISTEN
tcp6 0 0 ::1:953 :::* LISTEN
tcp6 0 256 ::ffff:192.168.178.2:22 ::ffff:192.168.178.5:55339 ESTABLISHED
tcp6 0 0 ::ffff:192.168.178.2:59299 ::ffff:192.168.178.2:389 TIME_WAIT
tcp6 0 0 ::1:33750 ::1:59639 TIME_WAIT
udp 0 0 0.0.0.0:2049 0.0.0.0:*
udp 105216 0 0.0.0.0:514 0.0.0.0:*
udp 0 0 0.0.0.0:32770 0.0.0.0:*
udp 0 0 192.168.178.2:137 0.0.0.0:*
udp 0 0 0.0.0.0:137 0.0.0.0:*
udp 0 0 192.168.178.2:138 0.0.0.0:*
udp 0 0 0.0.0.0:138 0.0.0.0:*
udp 0 0 0.0.0.0:922 0.0.0.0:*
udp 0 0 0.0.0.0:32801 0.0.0.0:*
udp 0 0 192.168.178.2:53 0.0.0.0:*
udp 0 0 127.0.0.1:53 0.0.0.0:*
udp 0 0 0.0.0.0:67 0.0.0.0:*
udp 0 0 0.0.0.0:69 0.0.0.0:*
udp 0 0 0.0.0.0:987 0.0.0.0:*
udp 0 0 0.0.0.0:990 0.0.0.0:*
udp 0 0 0.0.0.0:111 0.0.0.0:*
udp6 0 0 :::32802 :::*
udp6 0 0 :::53 :::*
raw 0 0 0.0.0.0:1 0.0.0.0:* 7
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node Path
unix 2 [ ] DGRAM 5335 @/org/kernel/udev/ude
vd
unix 8 [ ] DGRAM 323051 /dev/log
unix 2 [ ] DGRAM 931414
unix 3 [ ] STREAM CONNECTED 931413
unix 3 [ ] STREAM CONNECTED 931412
unix 2 [ ] DGRAM 930560
unix 2 [ ] DGRAM 325021
unix 2 [ ] DGRAM 324343
unix 2 [ ] DGRAM 323446
unix 2 [ ] DGRAM 323098
unix 2 [ ] DGRAM 262510
unix 2 [ ] DGRAM 9624

aswell as a nmap to the localhost:

root@poseidon:~# nmap localhost

Starting Nmap 4.20 ( Insecure.Org - Nmap Free Security Scanner, Tools & Hacking resources ) at 2008-03-25 23:52 CET
Interesting ports on localhost (127.0.0.1):
Not shown: 1688 closed ports
PORT STATE SERVICE
22/tcp open ssh
53/tcp open domain
111/tcp open rpcbind
139/tcp open netbios-ssn
445/tcp open microsoft-ds
925/tcp open unknown
953/tcp open rndc
993/tcp open imaps
2049/tcp open nfs

Nmap finished: 1 IP address (1 host up) scanned in 0.148 seconds
root@poseidon:~# nmap wglatina.dyndns.org

Starting Nmap 4.20 ( Insecure.Org - Nmap Free Security Scanner, Tools & Hacking resources ) at 2008-03-25 23:53 CET
Interesting ports on poseidon.wglatina.dyndns.org (192.168.178.2):
Not shown: 1689 closed ports
PORT STATE SERVICE
22/tcp open ssh
53/tcp open domain
111/tcp open rpcbind
139/tcp open netbios-ssn
445/tcp open microsoft-ds
925/tcp open unknown
993/tcp open imaps
2049/tcp open nfs

Nmap finished: 1 IP address (1 host up) scanned in 0.147 seconds
root@poseidon:~#

[gargola]

There is also something very curious when zimbra is running:
Notice the difference between nmap localhost and nmap wglatina.dyndns.org:

root@poseidon:~# nmap localhost

Starting Nmap 4.20 ( Insecure.Org - Nmap Free Security Scanner, Tools & Hacking resources ) at 2008-03-26 00:04 CET
Interesting ports on localhost (127.0.0.1):
Not shown: 1686 closed ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
53/tcp open domain
111/tcp open rpcbind
139/tcp open netbios-ssn
445/tcp open microsoft-ds
465/tcp open smtps
925/tcp open unknown
953/tcp open rndc
993/tcp open imaps
2049/tcp open nfs

Nmap finished: 1 IP address (1 host up) scanned in 0.167 seconds
root@poseidon:~# nmap wglatina.dyndns.org

Starting Nmap 4.20 ( Insecure.Org - Nmap Free Security Scanner, Tools & Hacking resources ) at 2008-03-26 00:05 CET
Interesting ports on poseidon.wglatina.dyndns.org (192.168.178.2):
Not shown: 1686 closed ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
53/tcp open domain
111/tcp open rpcbind
139/tcp open netbios-ssn
389/tcp open ldap
445/tcp open microsoft-ds
465/tcp open smtps
925/tcp open unknown
993/tcp open imaps
2049/tcp open nfs

Nmap finished: 1 IP address (1 host up) scanned in 0.161 seconds
root@poseidon:~#

During this nmap the problem described persits.

zimbra@poseidon:~$ zmcontrol status
Host wglatina.dyndns.org
antispam Running
antivirus Running
ldap Running
logger Running
mailbox Stopped
zmmailboxdctl is not running
mta Running
snmp Running
spell Running
stats Running
zimbra@poseidon:~$

[uxbod]

In your first netstat the following is present :-

tcp 0 0 0.0.0.0:993 0.0.0.0:* LISTEN

which is the IMAPS port that Zimbra users. Therefore, there already is a IMAP service running. Please check /etc/init.d for the imap start script, and shut it down and then try restarting Zimbra.

[gargola]

Hi,

The the imaps port remains used by some process after zimbra has been stopped (zmcontrol shutdown)

I do not have any other server (such as dovecot) installed. As such I had to find the owner of the process using the port with "fuser -n tcp 993"

Zimbra started working after restarting the service again.

I will report after rebooting the machine.

Is there anyway to know why this is happening?

regards,
gargola

[uxbod]

So what you are saying is that all the ZM processes did not shutdown ? Please check your logfiles when shutdown again to see if any error messages - Log Files - Zimbra :: Wiki