Possible corrupt LDAP

[weezbunbun]

A power failure in the server closet at our school corrupted our Zimbra server and it won't boot now. I setup zimbra on a new server and copied /opt/zimbra from the old machine with a live cd. It is Zimbra 7.1.1 open source edition on centos 5.7. When I go to start with the copied data I get the following errors-

tarting ldap...Done.
Failed.
/opt/zimbra/bin/ldap: line 56: kill: (4067) - Operation not permitted
/opt/zimbra/openldap/sbin/slapd: error while loading shared libraries: libdb-4.7.so: cannot open shared object file: No such file or directory
/opt/zimbra/bin/ldap: line 56: kill: (4067) - Operation not permitted
/opt/zimbra/openldap/sbin/slapd: error while loading shared libraries: libdb-4.7.so: cannot open shared object file: No such file or directory
/opt/zimbra/bin/ldap: line 56: kill: (4067) - Operation not permitted
/opt/zimbra/openldap/sbin/slapd: error while loading shared libraries: libdb-4.7.so: cannot open shared object file: No such file or directory
/opt/zimbra/bin/ldap: line 56: kill: (4067) - Operation not permitted
/opt/zimbra/openldap/sbin/slapd: error while loading shared libraries: libdb-4.7.so: cannot open shared object file: No such file or directory
/opt/zimbra/bin/ldap: line 56: kill: (4067) - Operation not permitted
/opt/zimbra/openldap/sbin/slapd: error while loading shared libraries: libdb-4.7.so: cannot open shared object file: No such file or directory
/opt/zimbra/bin/ldap: line 56: kill: (4067) - Operation not permitted
/opt/zimbra/openldap/sbin/slapd: error while loading shared libraries: libdb-4.7.so: cannot open shared object file: No such file or directory
/opt/zimbra/bin/ldap: line 56: kill: (4067) - Operation not permitted
/opt/zimbra/openldap/sbin/slapd: error while loading shared libraries: libdb-4.7.so: cannot open shared object file: No such file or directory
Failed to start slapd. Attempting debug start to determine error.
/opt/zimbra/openldap/sbin/slapd: error while loading shared libraries: libdb-4.7.so: cannot open shared object file: No such file or directory

I did do zmfixperms -extended, I can't seem to find anything in the forums that is similar to my problem so I am asking for help. I did read one thread that said if I had the data store and database I could rebuild but it didn't say how. Is this possible?
TIA

[phoenix]

There's a missing library according to that output, I'd suggest you run an 'upgrade' install of the same version of Zimbra to get everything reinstalled.

[weezbunbun]

When I do the upgrade I get the following-
Do you want to verify message store database integrity? [Y]
Verifying integrity of message store databases. This may take a while.
can't run /opt/zimbra/mysql/bin/mysqlcheck --defaults-file=/opt/zimbra/conf/my.cnf -S /opt/zimbra/db/mysql.sock -A -C -s -u root --password=L3a4uMqvB4CVyNejHbsB3x4dE --auto-repair: Not a directory

Then when I continue I get-
Backing up the ldap database...failed.

Then I get kicked out to a command prompt.

Am I hosed?

[quanah]

Can you try a software-only install of the same version on another box at the same OS, and copy the /opt/zimbra/data/ldap directory that you have backed up onto that box, and then use zmslapcat to export your LDIF file? That would at least give you the LDAP data you can use to restore.

[weezbunbun]

Quanah- I took your advice and exported then followed the 32 to 64 bit migration how to. The server now starts without errors but when I do a zmcontrol status I get
zmmailboxctl is not running

and /var/log/mail.log says-
Feb 6 22:36:07 mail zmmailboxdmgr[9625]: status requested
Feb 6 22:36:07 mail zmmailboxdmgr[9625]: stale pid 5388 found in /opt/zimbra/log/zmmailboxd_manager.pid: No such process
Feb 6 22:36:07 mail zmmailboxdmgr[9625]: assuming no other instance is running
Feb 6 22:36:07 mail zmmailboxdmgr[9625]: file /opt/zimbra/log/zmmailboxd.pid does not exist
Feb 6 22:36:07 mail zmmailboxdmgr[9625]: assuming no other instance is running
Feb 6 22:36:07 mail zmmailboxdmgr[9625]: no manager process is running
My host file is fine. Any suggestions? could it be the IP address of this temp server is different than my original server?

[quanah]

You need to check /opt/zimbra/log files, not /var/log.

Did you set the zmlocalconfig values for the LDAP passwords to match what they were on the old system?

[weezbunbun]

I copied the .xml file to the new server. Maybe that was the screwup.

[weezbunbun]

Here is /opt/zimbra/zmconfigd.log
2012-02-07 07:25:24,453 zmconfigd CRITICAL [12337-MainThread] zmconfigd started on mail.XXXXXXXXXXX.net with loglevel=3 pid=12337
2012-02-07 07:25:24,486 zmconfigd WARNING [12337-MainThread] Fetching All configs
log4j:WARN No appenders could be found for logger (zimbra.ldap).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See Apache log4j 1.2 - Frequently Asked Technical Questions for more info.
2012-02-07 07:25:29,911 zmconfigd WARNING [12337-MainThread] All configs fetched in 5.43 seconds
2012-02-07 07:25:33,089 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/mailboxd/etc/jetty.xml with mode 440 (0.33 sec)
2012-02-07 07:25:33,157 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/conf/clamd.conf with mode 440 (0.06 sec)
2012-02-07 07:25:33,325 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/conf/log4j.properties with mode 440 (0.17 sec)
2012-02-07 07:25:33,403 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/conf/mta_milter_options with mode 440 (0.08 sec)
2012-02-07 07:25:33,500 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/postfix/conf/master.cf with mode 440 (0.10 sec)
2012-02-07 07:25:33,519 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/cyrus-sasl-2.1.23.3z/etc/saslauthd.conf with mode 440 (0.02 sec)
2012-02-07 07:25:34,269 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/mailboxd/etc/jetty.properties with mode 440 (0.73 sec)
2012-02-07 07:25:34,358 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/conf/spnego_java_options with mode 440 (0.01 sec)
2012-02-07 07:25:34,474 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/mailboxd/webapps/zimbraAdmin/WEB-INF/web.xml with mode 440 (0.12 sec)
2012-02-07 07:25:34,799 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/mailboxd/etc/krb5.ini with mode 440 (0.32 sec)
2012-02-07 07:25:34,838 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/conf/freshclam.conf with mode 600 (0.04 sec)
2012-02-07 07:25:34,852 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/conf/stats.conf with mode 440 (0.01 sec)
2012-02-07 07:25:35,319 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/mailboxd/webapps/service/WEB-INF/web.xml with mode 440 (0.46 sec)
2012-02-07 07:25:35,332 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/conf/postfix_header_checks with mode 440 (0.01 sec)
2012-02-07 07:25:35,352 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/mailboxd/webapps/zimlet/WEB-INF/web.xml with mode 440 (0.02 sec)
2012-02-07 07:25:35,447 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/mailboxd/etc/spnego.conf with mode 440 (0.09 sec)
2012-02-07 07:25:35,655 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/mailboxd/webapps/zimbra/WEB-INF/web.xml with mode 440 (0.21 sec)
2012-02-07 07:25:35,661 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/mailboxd/etc/spnego.properties with mode 440 (0.00 sec)
2012-02-07 07:25:35,951 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/conf/amavisd.conf with mode 440 (0.29 sec)
2012-02-07 07:25:35,970 zmconfigd WARNING [12337-rewrites] Rewrote: /opt/zimbra/conf/salocal.cf with mode 440 (0.02 sec)
2012-02-07 07:25:38,486 zmconfigd WARNING [12337-MainThread] All rewrite threads completed in 5.73 sec
2012-02-07 07:25:38,489 zmconfigd WARNING [12337-MainThread] All restarts completed in 0.00 sec
2012-02-07 07:26:38,657 zmconfigd WARNING [12337-MainThread] Fetching All configs
2012-02-07 07:26:40,434 zmconfigd WARNING [12337-MainThread] All configs fetched in 1.78 seconds
2012-02-07 07:26:41,348 zmconfigd CRITICAL [12337-MainThread] Tracking service antispam
2012-02-07 07:26:41,349 zmconfigd WARNING [12337-MainThread] Watchdog: service antispam now available for watchdog.
2012-02-07 07:26:41,743 zmconfigd CRITICAL [12337-MainThread] Tracking service antivirus
2012-02-07 07:26:41,744 zmconfigd WARNING [12337-MainThread] Watchdog: service antivirus now available for watchdog.
2012-02-07 07:26:41,878 zmconfigd CRITICAL [12337-MainThread] Tracking service ldap
2012-02-07 07:26:41,881 zmconfigd WARNING [12337-MainThread] Watchdog: service ldap now available for watchdog.
2012-02-07 07:26:41,937 zmconfigd CRITICAL [12337-MainThread] Tracking service logger
2012-02-07 07:26:41,938 zmconfigd WARNING [12337-MainThread] Watchdog: service logger now available for watchdog.
2012-02-07 07:26:42,075 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmstorectl status norewrite returned 1 (30 - 13) (0.14 sec)
2012-02-07 07:26:42,076 zmconfigd CRITICAL [12337-MainThread] Tracking service mailbox
2012-02-07 07:26:42,381 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmmailboxdctl status norewrite returned 1 (25 - 13) (0.30 sec)
2012-02-07 07:26:42,381 zmconfigd CRITICAL [12337-MainThread] Tracking service mailboxd
2012-02-07 07:26:44,812 zmconfigd CRITICAL [12337-MainThread] Tracking service mta
2012-02-07 07:26:44,813 zmconfigd WARNING [12337-MainThread] Watchdog: service mta now available for watchdog.
2012-02-07 07:26:44,835 zmconfigd CRITICAL [12337-MainThread] Tracking service sasl
2012-02-07 07:26:44,835 zmconfigd WARNING [12337-MainThread] Watchdog: service sasl now available for watchdog.
2012-02-07 07:26:45,111 zmconfigd CRITICAL [12337-MainThread] Tracking service snmp
2012-02-07 07:26:45,114 zmconfigd WARNING [12337-MainThread] Watchdog: service snmp now available for watchdog.
2012-02-07 07:26:46,230 zmconfigd CRITICAL [12337-MainThread] Tracking service spell
2012-02-07 07:26:46,233 zmconfigd WARNING [12337-MainThread] Watchdog: service spell now available for watchdog.
2012-02-07 07:26:47,167 zmconfigd CRITICAL [12337-MainThread] Tracking service stats
2012-02-07 07:26:47,168 zmconfigd WARNING [12337-MainThread] Watchdog: service stats now available for watchdog.
2012-02-07 07:26:47,295 zmconfigd WARNING [12337-MainThread] Watchdog: service antivirus status is OK.
2012-02-07 07:26:47,911 zmconfigd WARNING [12337-MainThread] All rewrite threads completed in 0.00 sec
2012-02-07 07:26:47,913 zmconfigd WARNING [12337-MainThread] All restarts completed in 0.00 sec
Starting zmconfigd...zmconfigd is already running.
2012-02-07 07:27:48,224 zmconfigd WARNING [12337-MainThread] Fetching All configs
2012-02-07 07:27:48,595 zmconfigd WARNING [12337-MainThread] All configs fetched in 0.37 seconds
2012-02-07 07:27:49,079 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmstorectl status norewrite returned 1 (30 - 13) (0.06 sec)
2012-02-07 07:27:49,081 zmconfigd CRITICAL [12337-MainThread] Tracking service mailbox
2012-02-07 07:27:49,158 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmmailboxdctl status norewrite returned 1 (25 - 13) (0.08 sec)
2012-02-07 07:27:49,161 zmconfigd CRITICAL [12337-MainThread] Tracking service mailboxd
2012-02-07 07:27:50,618 zmconfigd WARNING [12337-MainThread] Watchdog: service antivirus status is OK.
2012-02-07 07:27:50,993 zmconfigd WARNING [12337-MainThread] All rewrite threads completed in 0.00 sec
2012-02-07 07:27:50,994 zmconfigd WARNING [12337-MainThread] All restarts completed in 0.00 sec
Starting zmconfigd...zmconfigd is already running.
2012-02-07 07:28:51,131 zmconfigd WARNING [12337-MainThread] Fetching All configs
2012-02-07 07:28:51,453 zmconfigd WARNING [12337-MainThread] All configs fetched in 0.32 seconds
2012-02-07 07:28:51,891 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmstorectl status norewrite returned 1 (30 - 13) (0.06 sec)
2012-02-07 07:28:51,892 zmconfigd CRITICAL [12337-MainThread] Tracking service mailbox
2012-02-07 07:28:51,933 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmmailboxdctl status norewrite returned 1 (25 - 13) (0.04 sec)
2012-02-07 07:28:51,934 zmconfigd CRITICAL [12337-MainThread] Tracking service mailboxd
2012-02-07 07:28:53,285 zmconfigd WARNING [12337-MainThread] Watchdog: service antivirus status is OK.
2012-02-07 07:28:53,463 zmconfigd WARNING [12337-MainThread] All rewrite threads completed in 0.00 sec
2012-02-07 07:28:53,466 zmconfigd WARNING [12337-MainThread] All restarts completed in 0.00 sec
Starting zmconfigd...zmconfigd is already running.
2012-02-07 07:29:53,589 zmconfigd WARNING [12337-MainThread] Fetching All configs
2012-02-07 07:29:53,913 zmconfigd WARNING [12337-MainThread] All configs fetched in 0.32 seconds
2012-02-07 07:29:54,309 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmstorectl status norewrite returned 1 (30 - 13) (0.06 sec)
2012-02-07 07:29:54,309 zmconfigd CRITICAL [12337-MainThread] Tracking service mailbox
2012-02-07 07:29:54,346 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmmailboxdctl status norewrite returned 1 (25 - 13) (0.03 sec)
2012-02-07 07:29:54,348 zmconfigd CRITICAL [12337-MainThread] Tracking service mailboxd
2012-02-07 07:29:55,721 zmconfigd WARNING [12337-MainThread] Watchdog: service antivirus status is OK.
2012-02-07 07:29:55,884 zmconfigd WARNING [12337-MainThread] All rewrite threads completed in 0.00 sec
2012-02-07 07:29:55,884 zmconfigd WARNING [12337-MainThread] All restarts completed in 0.00 sec
Starting zmconfigd...zmconfigd is already running.
2012-02-07 07:30:56,096 zmconfigd WARNING [12337-MainThread] Fetching All configs
2012-02-07 07:30:56,638 zmconfigd WARNING [12337-MainThread] All configs fetched in 0.54 seconds
2012-02-07 07:30:57,229 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmstorectl status norewrite returned 1 (30 - 13) (0.07 sec)
2012-02-07 07:30:57,230 zmconfigd CRITICAL [12337-MainThread] Tracking service mailbox
2012-02-07 07:30:57,266 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmmailboxdctl status norewrite returned 1 (25 - 13) (0.03 sec)
2012-02-07 07:30:57,266 zmconfigd CRITICAL [12337-MainThread] Tracking service mailboxd
2012-02-07 07:30:58,759 zmconfigd WARNING [12337-MainThread] Watchdog: service antivirus status is OK.
2012-02-07 07:30:58,974 zmconfigd WARNING [12337-MainThread] All rewrite threads completed in 0.00 sec
2012-02-07 07:30:58,976 zmconfigd WARNING [12337-MainThread] All restarts completed in 0.00 sec
Starting zmconfigd...zmconfigd is already running.
2012-02-07 07:31:59,125 zmconfigd WARNING [12337-MainThread] Fetching All configs
2012-02-07 07:31:59,424 zmconfigd WARNING [12337-MainThread] All configs fetched in 0.30 seconds
2012-02-07 07:31:59,809 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmstorectl status norewrite returned 1 (30 - 13) (0.06 sec)
2012-02-07 07:31:59,809 zmconfigd CRITICAL [12337-MainThread] Tracking service mailbox
2012-02-07 07:31:59,844 zmconfigd ERROR [12337-MainThread] Executed: /opt/zimbra/bin/zmmailboxdctl status norewrite returned 1 (25 - 13) (0.03 sec)
2012-02-07 07:31:59,845 zmconfigd CRITICAL [12337-MainThread] Tracking service mailboxd
2012-02-07 07:32:01,346 zmconfigd WARNING [12337-MainThread] Watchdog: service antivirus status is OK.
2012-02-07 07:32:01,571 zmconfigd WARNING [12337-MainThread] All rewrite threads completed in 0.00 sec
2012-02-07 07:32:01,572 zmconfigd WARNING [12337-MainThread] All restarts completed in 0.00 sec
2012-02-07 07:32:03,266 zmconfigd WARNING [12337-SIGTERM handler] Shutting down. Received signal 15

[quanah]

Yes, you need the localconfig.xml file. You'd also want the original certs off the old server if you can get them, as SSL/TLS is not likely to work correctly until you have them.

zmconfigd log doesn't show anything particularly interesting other than zmmailboxdctl & zmstorectl are failing. You can try running the commands manually to see what they output:

zmstorectl status norewrite
zmmailboxdctl status norewrite

[weezbunbun]

Are the certs in /opt/zimbra/conf? and can I just copy them from the old data or do I have to do some kind of import?

zmstorectl says zmmailboxctl is not running
zmmailboxdctl says mailboxd is not running