[SOLVED] Help!

[jreinhart]

Server down!

Please help...I've been working solid for over 24 hours and could use some insight.

I'm running ZCS 5.0.9 community edition on Ubuntu 8.04 LTS. Everything has been peachy, until...

We had some funkiness happen on Sunday...somehow another mail server got on our Zimbra server, and of course it stopped responding. I created a backup of the entire /opt/zimbra folder and proceeded to troubleshoot.

After all was said and done, I needed to restore from that backup.

I did the following:

Reinstall Ubuntu and reinstall Bind9 with Zimbra as a blank installation
zmcontrol stop
delete the "empty" zimbra install folder at /opt/zimbra
copy the backed up version from external drive to /opt/zimbra
run ./install.sh and tell it yes when it asks to do an upgrade

Per my research, this is the way to go. The "upgrade" shows the following:

root@mail:~/zcs-5.0.9_GA_2533.UBUNTU8.20080814162822# ./install.sh

Operations logged to /tmp/install.log.6963
Checking for existing installation...
zimbra-ldap...FOUND zimbra-ldap-5.0.9_GA_2533.UBUNTU8
zimbra-logger...FOUND zimbra-logger-5.0.9_GA_2533.UBUNTU8
zimbra-mta...FOUND zimbra-mta-5.0.9_GA_2533.UBUNTU8
zimbra-snmp...FOUND zimbra-snmp-5.0.9_GA_2533.UBUNTU8
zimbra-store...FOUND zimbra-store-5.0.9_GA_2533.UBUNTU8
zimbra-apache...FOUND zimbra-apache-5.0.9_GA_2533.UBUNTU8
zimbra-spell...FOUND zimbra-spell-5.0.9_GA_2533.UBUNTU8
zimbra-proxy...NOT FOUND
zimbra-archiving...NOT FOUND
zimbra-convertd...NOT FOUND
zimbra-cluster...NOT FOUND
zimbra-core...FOUND zimbra-core-5.0.9_GA_2533.UBUNTU8
ZCS upgrade from 5.0.9 to 5.0.9 will be performed.

Saving existing configuration file to /opt/zimbra/.saveconfig


PLEASE READ THIS AGREEMENT CAREFULLY BEFORE USING THE SOFTWARE.
ZIMBRA, INC. ("ZIMBRA") WILL ONLY LICENSE THIS SOFTWARE TO YOU IF YOU
FIRST ACCEPT THE TERMS OF THIS AGREEMENT. BY DOWNLOADING OR INSTALLING
THE SOFTWARE, OR USING THE PRODUCT, YOU ARE CONSENTING TO BE BOUND BY
THIS AGREEMENT. IF YOU DO NOT AGREE TO ALL OF THE TERMS OF THIS
AGREEMENT, THEN DO NOT DOWNLOAD, INSTALL OR USE THE PRODUCT.

License Terms for the Zimbra Collaboration Suite:
http://www.zimbra.com/license/zimbra..._eula_2.1.html


Press Return to continue

Checking for prerequisites...
NPTL...FOUND
sudo...FOUND sudo-1.6.9p10-1ubuntu3.3
libidn11...FOUND libidn11-1.1-1
libpcre3...FOUND libpcre3-7.4-1ubuntu2.1
libgmp3c2...FOUND libgmp3c2-2:4.2.2+dfsg-1ubuntu2
libexpat1...FOUND libexpat1-2.0.1-0ubuntu1
libstdc++6...FOUND libstdc++6-4.2.3-2ubuntu7
libstdc++5...FOUND libstdc++5-1:3.3.6-15ubuntu6
libltdl3...FOUND libltdl3-1.5.26-1ubuntu1
Prerequisite check complete.
Checking for standard system perl...
perl-5.8.8...FOUND standard system perl-5.8.8
Checking current number of databases...

Do you want to verify message store database integrity? [Y] N

Checking for installable packages

Found zimbra-core
Found zimbra-ldap
Found zimbra-logger
Found zimbra-mta
Found zimbra-snmp
Found zimbra-store
Found zimbra-apache
Found zimbra-spell
Found zimbra-proxy


The Zimbra Collaboration Suite appears already to be installed.
It can be upgraded with no effect on existing accounts,
or the current installation can be completely removed prior
to installation for a clean install.

Do you wish to upgrade? [Y]

Select the packages to install
Upgrading zimbra-core
Upgrading zimbra-ldap
Upgrading zimbra-logger
Upgrading zimbra-mta
Upgrading zimbra-snmp
Upgrading zimbra-store
Upgrading zimbra-apache
Upgrading zimbra-spell

Install zimbra-proxy [N]
Checking required space for zimbra-core
checking space for zimbra-store

Installing:
zimbra-core
zimbra-ldap
zimbra-logger
zimbra-mta
zimbra-snmp
zimbra-store
zimbra-apache
zimbra-spell

The system will be modified. Continue? [N] Y

Shutting down zimbra mail

Backing up the ldap database...done.

Removing existing packages

zimbra-ldap...done
zimbra-logger...done
zimbra-mta...done
zimbra-snmp...done
zimbra-store...done
zimbra-spell...done
zimbra-apache...done
zimbra-core...done

Removing deployed webapp directories
Installing packages

zimbra-core......zimbra-core_5.0.9_GA_2533.UBUNTU8_i386.deb...done
zimbra-ldap......zimbra-ldap_5.0.9_GA_2533.UBUNTU8_i386.deb...done
zimbra-logger......zimbra-logger_5.0.9_GA_2533.UBUNTU8_i386.deb...done
zimbra-mta......zimbra-mta_5.0.9_GA_2533.UBUNTU8_i386.deb...done
zimbra-snmp......zimbra-snmp_5.0.9_GA_2533.UBUNTU8_i386.deb...done
zimbra-store......zimbra-store_5.0.9_GA_2533.UBUNTU8_i386.deb...done
zimbra-apache......zimbra-apache_5.0.9_GA_2533.UBUNTU8_i386.deb...done
zimbra-spell......zimbra-spell_5.0.9_GA_2533.UBUNTU8_i386.deb...done

Setting defaults from saved config in /opt/zimbra/.saveconfig/config.save
HOSTNAME=localhost
LDAPHOST=
LDAPPORT=
SNMPTRAPHOST=
SMTPSOURCE=
SMTPDEST=
SNMPNOTIFY=0
SMTPNOTIFY=0
LDAPROOTPW=zimbra
LDAPZIMBRAPW=zimbra
LDAPPOSTPW=zmpostfix
LDAPREPPW=zmreplica
LDAPAMAVISPW=zmamavis
Restoring existing configuration file from /opt/zimbra/.saveconfig/config.save...done
Operations logged to /tmp/zmsetup.01001900-0000139029464.log
ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException localhost:389)
ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException localhost:389)
Setting defaults...ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException localhost:389)
ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException localhost:389)
done.
Upgrading from to 5.0.9_GA_2533
Stopping zimbra services
Verifying /opt/zimbra/conf/my.cnf
Starting mysql


And it hangs at that last point, indefinitely.

I've run these commands in an attempt to fix this (what appears to be a permissions issue):

chown -R zimbra:zimbra /opt/zimbra/*
/opt/zimbra/libexec/zmfixperms

Please help! We're dead in the water and this is a very important client.

Thanks,
- Jesse

Please help! I'm going to try to troubleshoot more but any assistance would be appreciated.

[jreinhart]

Anyone?

I'd really appreciate any direction or help on this.

- Jesse

[mmorse]

-Use ./install.sh -s (to install but not configure, then empty and copy in your backup - see it in action at step 3 here: Moving ZCS to Another Server » Zimbra :: Blog)
-The fact that you have "HOSTNAME=localhost" makes me curious as to the contents of /etc/hosts.

[ArcaneMagus]

You said you installed BIND9 on this server, are you trying to run a local caching DNS server on that machine? Or just a local server to redirect results for it?

I too think there is something wrong in your /etc/hosts or possibly whatever that local DNS server you setup is returning (If this server is set to use itself for queries).

[jreinhart]

I'm running Bind9 on the server so it can resolve queries to it's own e-mail server, no other reason. It only hosts the zone for the domain the mail is in so that user e-mails stay internal rather than the MX resolution going on the internet...I suppose there's not really a huge reason to run Bind, it's just how I've always done it.

Is there any way to compare what the hosts file was on the previous installation to what it is now? By this time I've reinstalled Ubuntu and the hosts file might be different. Here's what I have in my current hosts file:

127.0.0.1 localhost
192.168.100.5 mail.localdomain.com mail.localdomain.com

Now that I'm looking at it I know that the second line should read:

192.168.100.5 mail.localdomain.com mail

Strangely enough, when I type "host" in the terminal it returns with "mail" and when I type "hostname --fqdn" it returns "mail.localdomain.com...which is correct but the file seems wonky. So I'll change it appropriately...but do you know if I should remove the 127.0.0.1 localhost? Or edit it?

Thanks so much!

- Jesse

[mmorse]

Leave that line, but make it look like:
127.0.0.1 localhost.localdomain localhost
192.168.100.5 mail.domain.com mail

From the ZCS box/LAN running dig mail.domain.com / nslookup should return your 192 IP.

From your other thread [SOLVED] Installation complete on Ubuntu, problems receiving mail... it sounds like you're not going to complete this split DNS senario & recieve mail on port 25? (Where from the outside dig mx lookups would return the public IP.)

Originally Posted by jreinhart

then have each user set up their Zimbra account to pull down the mail from the pop3 server. So far this has been flawless.

[jreinhart]

OK, so just to clarify, my domain is "mydomain" my hosts should look like:

127.0.0.1 localhost.mydomain localhost
192.168.100.5 mail.mydomain.com mail

Is that right?

Yes it is split DNS - internal users use the BIND9 server on 192.168.100.5 to resolve ONLY requests to our domain name...BIND is set up with forwarders to forward other requests outside. However, in that scenario, it was really "split mail" with the POP server still retrieving mail from external senders and the Zimbra accounts pulling mail from that POP server. In my current scenario, the Zimbra does everything, including receiving mail from external and internal users.

External users resolve using public DNS servers, that resolve our domain name to our external IP, which then forwards mail traffic to 192.168.100.5 through a one-to-one NAT. FYI this whole setup was working fine until Sunday night when somehow the Exim4 daemon got on our server and screwed stuff up. By the time I realized what was causing the MTA not to start I think I had messed around too much to easily recover and decided to restore from backup...which I thought would be easier and faster...oops!

- Jesse

[mmorse]

-Hosts file looks correct.
-Ok so you did end up continuing on to having it receive inbound mail as your mx record, cool.

[jreinhart]

So in hosts I don't need

127.0.0.1 localhost.mydomain.com localhost

and only

127.0.0.1 localhost.mydomain localhost

?

Sorry, I just want to make sure things are as they should be.

- Jesse

[ArcaneMagus]

I just did some research on the 127.0.0.1 line and apparently having the localhost.localdomain entry first can cause some problems with certain applications, as requests for where 127.0.0.1 return localhost.localdomain where most applications expect just localhost. Is there a particular reason that you have localhost.localdomain first rather then a line like:
127.0.0.1 localhost localhost.localdomain
Just wondering here...