[SOLVED] mailbox and apache not running

[rtwick]

I had been running zimbra 5.0.8 on an ubuntu 6.06 box for quite some time and everything was working. One of my coworkers changed something in the admin web panel and since then zimbra stopped working. He doesn't have any idea what he changed. Now, I can not connect via imap/pop/webmail nor to the web admin panel.

zmcontrol status shows

zimbra@mail:~/bin$ zmcontrol status
Host mail.utkalika.net
        antispam                Running
        antivirus               Running
        ldap                    Running
        logger                  Running
        mailbox                 Running
        mta                     Running
        snmp                    Running
        spell                   Running
        stats                   Running

zmmailboxdmgr stop/start gives following error

zimbra@mail:~/bin$ zmmailboxdctl stop
/opt/zimbra/bin/zmmailboxdctl: line 73:  1255 Quit                    sudo ${zimbra_home}/libexec/zmmailboxdmgr threaddump
Terminated

After few reboots, below is a snippet form /opt/zimbra/zimbra.log

Nov 13 21:02:39 mail postfix/smtpd[25873]: connect from bulk4.ingdirect.com[63.113.210.143]
Nov 13 21:02:39 mail postfix/smtpd[25873]: warning: connect to 127.0.0.1:60000: Connection refused
Nov 13 21:02:39 mail postfix/smtpd[25873]: warning: problem talking to server 127.0.0.1:60000: Connection refused
Nov 13 21:02:40 mail postfix/smtpd[25873]: warning: connect to 127.0.0.1:60000: Connection refused
Nov 13 21:02:40 mail postfix/smtpd[25873]: warning: problem talking to server 127.0.0.1:60000: Connection refused
Nov 13 21:02:40 mail postfix/smtpd[25873]: NOQUEUE: reject: RCPT from bulk4.ingdirect.com[63.113.210.143]: 451 4.3.5 Server configuration problem; from=<saver@ingdirect.com> to=<RTWICK-KW-INGDIRECT.4D44D0@utkalika.net> proto=ESMTP helo=<bulk4.ingdirect.com>
Nov 13 21:02:40 mail postfix/smtpd[25873]: disconnect from bulk4.ingdirect.com[63.113.210.143]
Nov 13 21:02:57 mail zmmailboxdmgr[26422]: status requested
Nov 13 21:02:57 mail zmmailboxdmgr[26422]: status OK
Nov 13 21:02:59 mail zmmailboxdmgr[26480]: status requested
Nov 13 21:02:59 mail zmmailboxdmgr[26480]: status OK
Nov 13 21:03:07 mail postfix/smtpd[25873]: connect from unknown[219.159.0.47]
Nov 13 21:03:07 mail postfix/smtpd[25873]: lost connection after CONNECT from unknown[219.159.0.47]
Nov 13 21:03:07 mail postfix/smtpd[25873]: disconnect from unknown[219.159.0.47]
Nov 13 21:03:30 mail postfix/qmgr[9144]: 50E8D3A0221: from=<zimbra@mail.utkalika.net>, size=1055, nrcpt=1 (queue active)
Nov 13 21:03:30 mail postfix/qmgr[9144]: F04D33A0224: from=<TaxACTOnline@email.taxact-online.com>, size=23596, nrcpt=1 (queue active)
Nov 13 21:03:30 mail postfix/lmtp[26519]: connect to mail.utkalika.net[69.162.76.187]: Connection refused (port 7025)
Nov 13 21:03:30 mail postfix/lmtp[26520]: connect to mail.utkalika.net[69.162.76.187]: Connection refused (port 7025)
Nov 13 21:03:30 mail postfix/lmtp[26519]: 50E8D3A0221: to=<admin@utkalika.net>, relay=none, delay=37752, delays=37752/0.04/0.01/0, dsn=4.4.1, status=deferred (connect to mail.utkalika.net[69.162.76.187]: Connection refused)
Nov 13 21:03:30 mail postfix/lmtp[26520]: F04D33A0224: to=<rtwick@utkalika.net>, relay=none, delay=33973, delays=33973/0.03/0.01/0, dsn=4.4.1, status=deferred (connect to mail.utkalika.net[69.162.76.187]: Connection refused)
Nov 13 21:03:50 mail postfix/postfix-script[26637]: stopping the Postfix mail system
Nov 13 21:03:50 mail postfix/master[9141]: terminating on signal 15
Nov 13 21:03:56 mail zmmailboxdmgr[26724]: threaddump requested
Nov 13 21:03:58 mail zmmailboxdmgr[26731]: stop requested
Nov 13 21:04:05 mail zmmailboxdmgr[26867]: status requested
Nov 13 21:04:05 mail zmmailboxdmgr[26867]: status OK
Nov 13 21:09:21 mail postgrey[3562]: Process Backgrounded
Nov 13 21:09:21 mail postgrey[3562]: 2008/11/13-21:09:21 postgrey (type Net::Server::Multiplex) starting! pid(3562)
Nov 13 21:09:21 mail postgrey[3562]: Binding to TCP port 60000 on host 127.0.0.1
Nov 13 21:09:21 mail postgrey[3562]: Setting gid to "109 109"
Nov 13 21:09:21 mail postgrey[3562]: Setting uid to "109"
Nov 13 21:10:01 mail postfix/postqueue[4921]: fatal: Queue report unavailable - mail system is down
Nov 13 21:10:07 mail zmmailboxdmgr[5527]: status requested
Nov 13 21:10:07 mail zmmailboxdmgr[5527]: status OK
Nov 13 21:10:17 mail zmmailboxdmgr[7419]: status requested
Nov 13 21:10:17 mail zmmailboxdmgr[7419]: status OK
Nov 13 21:10:48 mail postfix/postfix-script[12756]: warning: not owned by root: /opt/zimbra/postfix-2.4.7.5z/conf/main.cf
Nov 13 21:10:48 mail postfix/postfix-script[12769]: warning: damaged message: corrupt/D540E3A021C
Nov 13 21:10:48 mail postfix/postfix-script[12770]: starting the Postfix mail system
Nov 13 21:10:48 mail postfix/master[12771]: daemon started -- version 2.4.7, configuration /opt/zimbra/postfix-2.4.7.5z/conf
Nov 13 21:10:48 mail postfix/qmgr[12775]: AC9C23A0227: from=<promo@email.batteries.com>, size=36331, nrcpt=1 (queue active)
Nov 13 21:10:48 mail postfix/qmgr[12775]: 72B333A0220: from=<newsletters_EB6DF8CDCBC180C6C912DB5A764A0EFC@response.eweek.com>, size=11873, nrcpt=1 (queue active)
Nov 13 21:10:48 mail postfix/qmgr[12775]: 9936A3A0231: from=<info@kingdomrealtymailer.net>, size=10795, nrcpt=1 (queue active)
Nov 13 21:10:48 mail postfix/qmgr[12775]: D47FB3A022D: from=<info@newegg.com>, size=24291, nrcpt=1 (queue active)
Nov 13 21:10:48 mail postfix/qmgr[12775]: DB32A3A021A: from=<38R4cSQcTBnUghkXierVaXVdhnm.ZhhZeX.VhfkmpbVdnmdTebdT.gXm@cbg.bounces.google.com>, size=45380, nrcpt=1 (queue active)
Nov 13 21:10:48 mail postfix/qmgr[12775]: 63E9F3A022C: from=<info@newegg.com>, size=23349, nrcpt=1 (queue active)
Nov 13 21:10:48 mail postfix/lmtp[12784]: connect to mail.utkalika.net[69.162.76.187]: Connection refused (port 7025)
Nov 13 21:10:48 mail postfix/lmtp[12786]: connect to mail.utkalika.net[69.162.76.187]: Connection refused (port 7025)
Nov 13 21:10:48 mail postfix/lmtp[12787]: connect to mail.utkalika.net[69.162.76.187]: Connection refused (port 7025)
Nov 13 21:10:48 mail postfix/lmtp[12789]: connect to mail.utkalika.net[69.162.76.187]: Connection refused (port 7025)
Nov 13 21:10:48 mail postfix/lmtp[12791]: connect to mail.utkalika.net[69.162.76.187]: Connection refused (port 7025)
Nov 13 21:10:48 mail postfix/lmtp[12787]: 9936A3A0231: to=<rtwick@utkalika.net>, relay=none, delay=13132, delays=13132/0.04/0.01/0, dsn=4.4.1, status=deferred (connect to mail.utkalika.net[69.162.76.187]: Connection refused)
Nov 13 21:10:48 mail postfix/lmtp[12784]: AC9C23A0227: to=<rtwick-kw-batteries.com.bb0887@utkalika.net>, relay=none, delay=29605, delays=29605/0.08/0.01/0, dsn=4.4.1, status=deferred (connect to mail.utkalika.net[69.162.76.187]: Connection refused)
Nov 13 21:10:48 mail postfix/lmtp[12789]: D47FB3A022D: to=<rtwick@utkalika.net>, relay=none, delay=21649, delays=21649/0.05/0.01/0, dsn=4.4.1, status=deferred (connect to mail.utkalika.net[69.162.76.187]: Connection refused)
Nov 13 21:10:48 mail postfix/lmtp[12786]: 72B333A0220: to=<rtwick@utkalika.net>, relay=none, delay=38688, delays=38688/0.04/0.01/0, dsn=4.4.1, status=deferred (connect to mail.utkalika.net[69.162.76.187]: Connection refused)
Nov 13 21:10:48 mail postfix/lmtp[12791]: DB32A3A021A: to=<rtwick@utkalika.net>, relay=none, delay=48422, delays=48422/0.07/0.01/0, dsn=4.4.1, status=deferred (connect to mail.utkalika.net[69.162.76.187]: Connection refused)
Nov 13 21:10:48 mail postfix/qmgr[12775]: warning: connect to transport retry: No such file or directory
Nov 13 21:10:48 mail postfix/qmgr[12775]: 63E9F3A022C: to=<rtwick@utkalika.net>, relay=none, delay=21658, delays=21657/0.21/0/0, dsn=4.3.0, status=deferred (mail transport unavailable)
Nov 13 21:11:09 mail zmmailboxdmgr[13551]: status requested
Nov 13 21:11:09 mail zmmailboxdmgr[13551]: status OK
Nov 13 21:11:11 mail zmmailboxdmgr[13609]: status requested
Nov 13 21:11:11 mail zmmailboxdmgr[13609]: status OK
Nov 13 21:12:05 mail zmmailboxdmgr[13850]: status requested
Nov 13 21:12:05 mail zmmailboxdmgr[13850]: status OK
root@mail:~#

Any pointers where should I look ?

Thanks in advance

-R'twick

[uxbod]

One thing I would fix is to get postgrey starting before Zimbra

warning: connect to 127.0.0.1:60000: Connection refused

and as you can see from the log extract you get a few failure messages before

Nov 13 21:09:21 mail postgrey[3562]: Binding to TCP port 60000 on host 127.0.0.1

[rtwick]

Thanks uxbod.
I removed postgrey for now. However, that did not fix the problem. I still have the same problem .

nmap shows only smtp, smtps and ldap running. However, ps shows apache is running but it's not binding to port 80/443. Even admin ports are not open. same with pop and imap. They are also not running.

Is it possible to just rename /opt/zimbra directory, reinstall zimbra and after setting everything as it was before can I just move the messages back ?

[uxbod]

Have you disabled IPtables ? We could also check your DNS setup aswell

cat /etc/hosts
cat /etc/resolv.conf
dig yourdomainname mx
dig yourdomainname any
host `hostname` <- note backticks and not single quotes

you could also just shut everything down, tar up /opt/zimbra, and then install over the top of your current installation.

[rtwick]

Found the problem. Couple of .pid files in /opt/zimbra/log were owned by root instead of zimbra including zmmailboxmgr.pid. That prevented mailboxmgr to come up as user zimbra. Once thepermissions were fixed, everything was normal. Most like cause is that zimbra was restarted by user root somehow.

Thanks