Server goes down (Unable to determine enabled services from ldap)

[Solt]

Hi all,

My server goes down today and I found ldap is not working with captioned error, I searched the post and found it should be cert expired.

I follow the steps below, however "/opt/zimbra/bin/zmcertmgr deploycrt self" returns an error, can someone help please?

Steps:
# /opt/zimbra/bin/zmcertmgr createca -new
# /opt/zimbra/bin/zmcertmgr createcrt -new -days 365
# /opt/zimbra/bin/zmcertmgr deploycrt self
# /opt/zimbra/bin/zmcertmgr deployca
# su zimbra
$ /opt/zimbra/bin/zmcontrol restart


Error message
[root@ms1 ~]# /opt/zimbra/bin/zmcertmgr deploycrt self
** Saving server config key zimbraSSLCertificate...failed.
** Saving server config key zimbraSSLPrivateKey...failed.
** Installing mta certificate and key...done.
** Installing slapd certificate and key...done.
** Installing proxy certificate and key...done.
** Creating pkcs12 file /opt/zimbra/ssl/zimbra/jetty.pkcs12...done.
** Creating keystore file /opt/zimbra/mailboxd/etc/keystore...done.
** Installing CA to /opt/zimbra/conf/ca...unable to load certificate
10673:error:0906D06C:PEM routines:PEM_read_bio:no start lineem_lib.c:650:Expec ting: TRUSTED CERTIFICATE

[Solt]

zmcontrol status

[zimbra@ms1 opt]$ zmcontrol status
Unable to determine enabled services from ldap.
Enabled services read from cache. Service list may be inaccurate.
Host ms1.mydomain
antispam Running
antivirus Running
ldap Running
logger Stopped
zmlogswatchctl is not running
mailbox Stopped
zmmailboxdctl is not running.
memcached Running
mta Running
snmp Running
spell Running
stats Running

cat /etc/hosts

[zimbra@ms1 opt]$ cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
#::1 localhost6.localdomain6 localhost6
10.0.85.21 ms1.mydomain ms1
203.185.55.209 ms1.mydomain ms1

dig ms1.mydomain mx

[zimbra@ms1 opt]$ dig ms1.mydomain mx

; <<>> DiG 9.3.6-P1-RedHat-9.3.6-16.P1.el5_7.1 <<>> ms1.ipehk.com.hk mx
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39136
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;ms1.mydomain. IN MX

;; AUTHORITY SECTION:
mydomain. 3337 IN SOA ns1.mydomain. root.ns1.mydomain. 2 10800 3600 604800 3600

;; ADDITION SECTION:
ns1.mydomain IN NS 10.0.85.20
;; Query time: 2 msec
;; SERVER: 10.0.85.11#53(10.0.85.11)
;; WHEN: Thu Mar 29 11:07:10 2012
;; MSG SIZE rcvd: 79

dig ms1.mydomain any

[zimbra@ms1 opt]$ dig ms1.mydomain any

; <<>> DiG 9.3.6-P1-RedHat-9.3.6-16.P1.el5_7.1 <<>> ms1.mydomain any
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47307
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;ms1.mydomain. IN ANY

;; ANSWER SECTION:
ms1.mydomain. 3600 IN A 10.0.85.21

;; AUTHORITY SECTION:
mydomain. 3600 IN NS ns1.mydomain.

;; Query time: 24 msec
;; SERVER: 10.0.85.11#53(10.0.85.11)
;; WHEN: Thu Mar 29 11:08:31 2012
;; MSG SIZE rcvd: 68

[Solt]

Here is the error log from zimbra.log

Mar 29 10:34:48 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping All MTA Authentication Target URLs update.
Mar 29 10:34:48 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping getAllMtaAuthURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.net.ssl.SSLHandshakeException sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed)
Mar 29 10:34:53 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping Configuration for server ms1.ipehk.com.hk update.
Mar 29 10:34:53 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: gs:ms1.ipehk.com.hk ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.net.ssl.SSLHandshakeException sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed)
Mar 29 10:34:53 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Sleeping...Key lookup failed.
Mar 29 10:35:04 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping Global system configuration update.
Mar 29 10:35:04 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: gacf ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.net.ssl.SSLHandshakeException sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed)
Mar 29 10:35:09 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping All Reverse Proxy URLs update.
Mar 29 10:35:09 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping getAllReverseProxyURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.net.ssl.SSLHandshakeException sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed)
Mar 29 10:35:14 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping All Reverse Proxy Backends update.
Mar 29 10:35:14 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping getAllReverseProxyBackends ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.net.ssl.SSLHandshakeException sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed)
Mar 29 10:35:20 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping All Memcached Servers update.
Mar 29 10:35:20 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping getAllMemcachedServers ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.net.ssl.SSLHandshakeException sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed)
Mar 29 10:35:25 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping All MTA Authentication Target URLs update.
Mar 29 10:35:25 ms1 zimbramon[6546]: 6546:info: zmmtaconfig: Skipping getAllMtaAuthURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.net.ssl.SSLHandshakeException sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed)

[Solt]

host ms1.mydomain

ms1.mydomain has address 10.0.85.21

[phoenix]

According to the output you've posted there's no A or MX records for your server and your hosts file is incorrect. Go to the Split DNS article and read what's necessary for the hosts file and fix that before you go any further, you should also look at the 'Verfiry....' section of that article for the correct commands to check your configuration. When you've done that, follow the instructions in this article for recreating the certificates.

[Solt]

I have removed "203.185.55.209 ms1.mydomain ms1" from /etc/hosts, and the server ran for 1.5 years, no problem resolving the server address from internet or internal network.

ns1.mydomain is the name server we used, and any DNS lookup will be forwarded to this from our main dns.

[Solt]

May I know if there is any settings in Zimbra so that it has notification email when the cert if going to be expired?