Results 1 to 4 of 4

Thread: [SOLVED] System Fails after Deployment of Commercial SSL Cert (ZCS-6.0.1 on Ubutntu 8

  1. #1
    rwc101010 is offline Active Member
    Join Date
    Dec 2006
    Location
    SF Bay Area
    Posts
    30
    Rep Power
    8

    Default [SOLVED] System Fails after Deployment of Commercial SSL Cert (ZCS-6.0.1 on Ubutntu 8

    While testing the migration from ZCS 6.0.1 GA 32-bit to ZCS 6.0.1 GA 64-bit on a development machine, I encountered an issue after deploying a commercial SSL certificate. After what appears to be a successful deploy, restarting ZCS results in the following error:

    Unable to determine enabled services from ldap.
    Enabled services read from cache. Service list may be inaccurate.
    After restart, no services are functional, and lots of error messages stack up in the logs.

    To build this server, I performed a fresh install of Ubuntu 8 LTS Server, then fully patched the system and installed the required Zimbra libraries.

    Code:
    Checking for prerequisites...
         FOUND: NPTL
         FOUND: sudo-1.6.9p10-1ubuntu3.5
         FOUND: libidn11-1.1-1
         FOUND: libpcre3-7.4-1ubuntu2.1
         FOUND: libgmp3c2-2:4.2.2+dfsg-1ubuntu2
         FOUND: libexpat1-2.0.1-0ubuntu1
         FOUND: libstdc++6-4.2.4-1ubuntu4
         FOUND: libstdc++5-1:3.3.6-15ubuntu6
         FOUND: libperl5.8-5.8.8-12ubuntu0.4
    Checking for suggested prerequisites...
        FOUND: perl-5.8.8
        FOUND: sysstat
    Prerequisite check complete.
    /etc/hosts and local DNS seem to be setup properly, to provide the internal IP to the system for DNS lookups:

    Code:
    root@mail:~# hostname
    mail
    root@mail:~# hostname -f
    mail.xxxxxxx.xxx
    root@mail:~# dig -t mx xxxxxxx.xxx
    
    ; <<>> DiG 9.4.2-P2 <<>> -t mx xxxxxxx.xxx
    ;; global options:  printcmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62757
    ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 4, ADDITIONAL: 4
    
    ;; QUESTION SECTION:
    ;xxxxxxx.xxx.			IN	MX
    
    ;; ANSWER SECTION:
    xxxxxxx.xxx.		86400	IN	MX	15 mx2.xxxxxxx.xxx.
    xxxxxxx.xxx.		86400	IN	MX	10 mail.xxxxxxx.xxx.
    
    ;; AUTHORITY SECTION:
    xxxxxxx.xxx.		86400	IN	NS	ns1.xxxxxxx.xxx.
    xxxxxxx.xxx.		86400	IN	NS	ns2.xxxxxxx.xxx.
    xxxxxxx.xxx.		86400	IN	NS	ns2.xxxxxxx.yyy.
    xxxxxxx.xxx.		86400	IN	NS	ns1.xxxxxxx.yyy.
    
    ;; ADDITIONAL SECTION:
    mail.xxxxxxx.xxx.	86400	IN	A	192.168.3.5
    ns1.xxxxxxx.xxx.	78475	IN	A	~removed external IP~
    ns2.xxxxxxx.xxx.	74	IN	A	~removed external IP~
    ns1.xxxxxxx.yyy.	86400	IN	A	~removed external IP~
    ns2.xxxxxxx.yyy.	86400	IN	A	~removed external IP~
    
    ;; Query time: 0 msec
    ;; SERVER: 192.168.3.5#53(192.168.3.5)
    ;; WHEN: Fri Oct 16 02:01:16 2009
    ;; MSG SIZE  rcvd: 217
    Once the base platform was built, I performed a fresh install of 6.0.1_GA_1816.UBUNTU8_64 UBUNTU8_64 NETWORK edition, maintaining configuration parity with my 32-bit development system to be migrated - items such as the following were changed during the final install configuration to match the 32 bit system:

    • Administrative Account Password
    • SPAM User Account
    • HAM User Account
    • LDAP Root Password
    • LDAP Replication Password
    • LDAP Postfix Password
    • LDAP Amavis Password
    • LDAP Nginx Password
    • Default COS Settings


    After install, all services function and all interfaces are accessible.

    The next step on my migration checklist entails moving the Commercial cert from my 32-bit dev system, to my 64 bit dev system.

    I copied the original working commercial_ca.crt to /tmp/ssl/

    Code:
    	-----BEGIN CERTIFICATE-----
    	MIIE3jCCA8agAwIBAgICAwEwDQYJKoZIhvcNAQEFBQAwYzELMAkGA1UEBhMCVVMx
    	ITAfBgNVBAoTGFRoZSBHbyBEYWRkeSBHcm91cCwgSW5jLjExMC8GA1UECxMoR28g
    	RGFkZHkgQ2xhc3MgMiBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0wNjExMTYw
    	MTU0MzdaFw0yNjExMTYwMTU0MzdaMIHKMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH
    	QXJpem9uYTETMBEGA1UEBxMKU2NvdHRzZGFsZTEaMBgGA1UEChMRR29EYWRkeS5j
    	b20sIEluYy4xMzAxBgNVBAsTKmh0dHA6Ly9jZXJ0aWZpY2F0ZXMuZ29kYWRkeS5j
    	b20vcmVwb3NpdG9yeTEwMC4GA1UEAxMnR28gRGFkZHkgU2VjdXJlIENlcnRpZmlj
    	YXRpb24gQXV0aG9yaXR5MREwDwYDVQQFEwgwNzk2OTI4NzCCASIwDQYJKoZIhvcN
    	AQEBBQADggEPADCCAQoCggEBAMQt1RWMnCZM7DI161+4WQFapmGBWTtwY6vj3D3H
    	KrjJM9N55DrtPDAjhI6zMBS2sofDPZVUBJ7fmd0LJR4h3mUpfjWoqVTr9vcyOdQm
    	VZWt7/v+WIbXnvQAjYwqDL1CBM6nPwT27oDyqu9SoWlm2r4arV3aLGbqGmu75RpR
    	SgAvSMeYddi5Kcju+GZtCpyz8/x4fKL4o/K1w/O5epHBp+YlLpyo7RJlbmr2EkRT
    	cDCVw5wrWCs9CHRK8r5RsL+H0EwnWGu1NcWdrxcx+AuP7q2BNgWJCJjPOq8lh8BJ
    	6qf9Z/dFjpfMFDniNoW1fho3/Rb2cRGadDAW/hOUoz+EDU8CAwEAAaOCATIwggEu
    	MB0GA1UdDgQWBBT9rGEyk2xF1uLuhV+auud2mWjM5zAfBgNVHSMEGDAWgBTSxLDS
    	kdRMEXGzYcs9of7dqGrU4zASBgNVHRMBAf8ECDAGAQH/AgEAMDMGCCsGAQUFBwEB
    	BCcwJTAjBggrBgEFBQcwAYYXaHR0cDovL29jc3AuZ29kYWRkeS5jb20wRgYDVR0f
    	BD8wPTA7oDmgN4Y1aHR0cDovL2NlcnRpZmljYXRlcy5nb2RhZGR5LmNvbS9yZXBv
    	c2l0b3J5L2dkcm9vdC5jcmwwSwYDVR0gBEQwQjBABgRVHSAAMDgwNgYIKwYBBQUH
    	AgEWKmh0dHA6Ly9jZXJ0aWZpY2F0ZXMuZ29kYWRkeS5jb20vcmVwb3NpdG9yeTAO
    	BgNVHQ8BAf8EBAMCAQYwDQYJKoZIhvcNAQEFBQADggEBANKGwOy9+aG2Z+5mC6IG
    	OgRQjhVyrEp0lVPLN8tESe8HkGsz2ZbwlFalEzAFPIUyIXvJxwqoJKSQ3kbTJSMU
    	A2fCENZvD117esyfxVgqwcSeIaha86ykRvOe5GPLL5CkKSkB2XIsKd83ASe8T+5o
    	0yGPwLPk9Qnt0hCqU7S+8MxZC9Y7lhyVJEnfzuz9p0iRFEUOOjZv2kWzRaJBydTX
    	RE4+uXR21aITVSzGh6O1mawGhId/dQb8vxRMDsxuxN89txJx9OjxUUAiKEngHUuH
    	qDTMBqLdElrRhjZkAzVvb3du6/KFUJheqwNTrZEjYx8WnM25sgVjOuH0aBsXBTWV
    	U+4=
    	-----END CERTIFICATE-----
    	-----BEGIN CERTIFICATE-----
    	MIIE+zCCBGSgAwIBAgICAQ0wDQYJKoZIhvcNAQEFBQAwgbsxJDAiBgNVBAcTG1Zh
    	bGlDZXJ0IFZhbGlkYXRpb24gTmV0d29yazEXMBUGA1UEChMOVmFsaUNlcnQsIElu
    	Yy4xNTAzBgNVBAsTLFZhbGlDZXJ0IENsYXNzIDIgUG9saWN5IFZhbGlkYXRpb24g
    	QXV0aG9yaXR5MSEwHwYDVQQDExhodHRwOi8vd3d3LnZhbGljZXJ0LmNvbS8xIDAe
    	BgkqhkiG9w0BCQEWEWluZm9AdmFsaWNlcnQuY29tMB4XDTA0MDYyOTE3MDYyMFoX
    	DTI0MDYyOTE3MDYyMFowYzELMAkGA1UEBhMCVVMxITAfBgNVBAoTGFRoZSBHbyBE
    	YWRkeSBHcm91cCwgSW5jLjExMC8GA1UECxMoR28gRGFkZHkgQ2xhc3MgMiBDZXJ0
    	aWZpY2F0aW9uIEF1dGhvcml0eTCCASAwDQYJKoZIhvcNAQEBBQADggENADCCAQgC
    	ggEBAN6d1+pXGEmhW+vXX0iG6r7d/+TvZxz0ZWizV3GgXne77ZtJ6XCAPVYYYwhv
    	2vLM0D9/AlQiVBDYsoHUwHU9S3/Hd8M+eKsaA7Ugay9qK7HFiH7Eux6wwdhFJ2+q
    	N1j3hybX2C32qRe3H3I2TqYXP2WYktsqbl2i/ojgC95/5Y0V4evLOtXiEqITLdiO
    	r18SPaAIBQi2XKVlOARFmR6jYGB0xUGlcmIbYsUfb18aQr4CUWWoriMYavx4A6lN
    	f4DD+qta/KFApMoZFv6yyO9ecw3ud72a9nmYvLEHZ6IVDd2gWMZEewo+YihfukEH
    	U1jPEX44dMX4/7VpkI+EdOqXG68CAQOjggHhMIIB3TAdBgNVHQ4EFgQU0sSw0pHU
    	TBFxs2HLPaH+3ahq1OMwgdIGA1UdIwSByjCBx6GBwaSBvjCBuzEkMCIGA1UEBxMb
    	VmFsaUNlcnQgVmFsaWRhdGlvbiBOZXR3b3JrMRcwFQYDVQQKEw5WYWxpQ2VydCwg
    	SW5jLjE1MDMGA1UECxMsVmFsaUNlcnQgQ2xhc3MgMiBQb2xpY3kgVmFsaWRhdGlv
    	biBBdXRob3JpdHkxITAfBgNVBAMTGGh0dHA6Ly93d3cudmFsaWNlcnQuY29tLzEg
    	MB4GCSqGSIb3DQEJARYRaW5mb0B2YWxpY2VydC5jb22CAQEwDwYDVR0TAQH/BAUw
    	AwEB/zAzBggrBgEFBQcBAQQnMCUwIwYIKwYBBQUHMAGGF2h0dHA6Ly9vY3NwLmdv
    	ZGFkZHkuY29tMEQGA1UdHwQ9MDswOaA3oDWGM2h0dHA6Ly9jZXJ0aWZpY2F0ZXMu
    	Z29kYWRkeS5jb20vcmVwb3NpdG9yeS9yb290LmNybDBLBgNVHSAERDBCMEAGBFUd
    	IAAwODA2BggrBgEFBQcCARYqaHR0cDovL2NlcnRpZmljYXRlcy5nb2RhZGR5LmNv
    	bS9yZXBvc2l0b3J5MA4GA1UdDwEB/wQEAwIBBjANBgkqhkiG9w0BAQUFAAOBgQC1
    	QPmnHfbq/qQaQlpE9xXUhUaJwL6e4+PrxeNYiY+Sn1eocSxI0YGyeR+sBjUZsE4O
    	WBsUs5iB0QQeyAfJg594RAoYC5jcdnplDQ1tgMQLARzLrUc+cb53S8wGd9D0Vmsf
    	SxOaFIqII6hR8INMqzW/Rn453HWkrugp++85j09VZw==
    	-----END CERTIFICATE-----
    	-----BEGIN CERTIFICATE-----
    	MIIC5zCCAlACAQEwDQYJKoZIhvcNAQEFBQAwgbsxJDAiBgNVBAcTG1ZhbGlDZXJ0
    	IFZhbGlkYXRpb24gTmV0d29yazEXMBUGA1UEChMOVmFsaUNlcnQsIEluYy4xNTAz
    	BgNVBAsTLFZhbGlDZXJ0IENsYXNzIDIgUG9saWN5IFZhbGlkYXRpb24gQXV0aG9y
    	aXR5MSEwHwYDVQQDExhodHRwOi8vd3d3LnZhbGljZXJ0LmNvbS8xIDAeBgkqhkiG
    	9w0BCQEWEWluZm9AdmFsaWNlcnQuY29tMB4XDTk5MDYyNjAwMTk1NFoXDTE5MDYy
    	NjAwMTk1NFowgbsxJDAiBgNVBAcTG1ZhbGlDZXJ0IFZhbGlkYXRpb24gTmV0d29y
    	azEXMBUGA1UEChMOVmFsaUNlcnQsIEluYy4xNTAzBgNVBAsTLFZhbGlDZXJ0IENs
    	YXNzIDIgUG9saWN5IFZhbGlkYXRpb24gQXV0aG9yaXR5MSEwHwYDVQQDExhodHRw
    	Oi8vd3d3LnZhbGljZXJ0LmNvbS8xIDAeBgkqhkiG9w0BCQEWEWluZm9AdmFsaWNl
    	cnQuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDOOnHK5avIWZJV16vY
    	dA757tn2VUdZZUcOBVXc65g2PFxTXdMwzzjsvUGJ7SVCCSRrCl6zfN1SLUzm1NZ9
    	WlmpZdRJEy0kTRxQb7XBhVQ7/nHk01xC+YDgkRoKWzk2Z/M/VXwbP7RfZHM047QS
    	v4dk+NoS/zcnwbNDu+97bi5p9wIDAQABMA0GCSqGSIb3DQEBBQUAA4GBADt/UG9v
    	UJSZSWI4OB9L+KXIPqeCgfYrx+jFzug6EILLGACOTb2oWH+heQC1u+mNr0HZDzTu
    	IYEZoDJJKPTEjlbVUjP9UNV+mWwD5MlM/Mtsq2azSiGM5bUMMj4QssxsodyamEwC
    	W/POuZ6lcg5Ktz885hZo+L7tdEy8W9ViH0Pd
    	-----END CERTIFICATE-----
    I copied the original working commercial.crt to /tmp/ssl/

    Code:
    	-----BEGIN CERTIFICATE-----
    	  ~~Data Removed From Post for Security~~
    	-----END CERTIFICATE-----
    I copied the original working commercial.key to /opt/zimbra/ssl/zimbra/commercial/

    Code:
    	-----BEGIN RSA PRIVATE KEY-----
    	  ~~Data Removed From Post for Security~~
    	-----END RSA PRIVATE KEY-----
    I placed the original working commercial.csr to /opt/zimbra/ssl/zimbra/commercial/

    Code:
    	-----BEGIN CERTIFICATE REQUEST-----
    	  ~~Data Removed From Post for Security~~
    	-----END CERTIFICATE REQUEST-----
    I then tested and deployed the Commercial SSL Certificate via the command line:

    Code:
    	root@mail:~# cd /tmp/ssl
    
    	root@mail:/tmp/ssl# /opt/zimbra/bin/zmcertmgr verifycrt comm /opt/zimbra/ssl/zimbra/commercial/commercial.key ./commercial.crt ./commercial_ca.crt
    	** Verifying ./commercial.crt against /opt/zimbra/ssl/zimbra/commercial/commercial.key
    	Certificate (./commercial.crt) and private key (/opt/zimbra/ssl/zimbra/commercial/commercial.key) match.
    	Valid Certificate: ./commercial.crt: OK
    
    	root@mail:/tmp/ssl# /opt/zimbra/bin/zmcertmgr deploycrt comm ./commercial.crt ./commercial_ca.crt
    	** Verifying ./commercial.crt against /opt/zimbra/ssl/zimbra/commercial/commercial.key
    	Certificate (./commercial.crt) and private key (/opt/zimbra/ssl/zimbra/commercial/commercial.key) match.
    	Valid Certificate: ./commercial.crt: OK
    	** Copying ./commercial.crt to /opt/zimbra/ssl/zimbra/commercial/commercial.crt
    	** Appending ca chain ./commercial_ca.crt to /opt/zimbra/ssl/zimbra/commercial/commercial.crt
    	** Saving server config key zimbraSSLCertificate...done.
    	** Saving server config key zimbraSSLPrivateKey...done.
    	** Installing mta certificate and key...done.
    	** Installing slapd certificate and key...done.
    	** Installing proxy certificate and key...done.
    	** Creating pkcs12 file /opt/zimbra/ssl/zimbra/jetty.pkcs12...done.
    	** Creating keystore file /opt/zimbra/mailboxd/etc/keystore...done.
    	** Installing CA to /opt/zimbra/conf/ca...done.
    The Commercial Certificate deployment seemed to complete without issue.

    I then restarted ZCS (and this is where the fun begins):

    Code:
    	zimbra@mail:~$ zmcontrol stop
    	Host mail.xxxxxxx.xxx
    		Stopping stats...Done.
    		Stopping mta...Done.
    		Stopping spell...Done.
    		Stopping snmp...Done.
    		Stopping archiving...Done.
    		Stopping antivirus...Done.
    		Stopping antispam...Done.
    		Stopping imapproxy...Done.
    		Stopping memcached...Done.
    		Stopping mailbox...Done.
    		Stopping convertd...Done.
    		Stopping logger...Done.
    		Stopping ldap...Done.
    
    	zimbra@mail:~$ zmcontrol start
    	Host mail.xxxxxxx.xxx
    		Starting ldap...Done.
    	Unable to determine enabled services from ldap.
    	Enabled services read from cache. Service list may be inaccurate.
    		Starting logger...Done.
    		Starting convertd...Done.
    		Starting mailbox...Done.
    		Starting memcached...Done.
    		Starting antispam...Done.
    		Starting antivirus...Done.
    		Starting snmp...Done.
    		Starting spell...Done.
    		Starting mta...Done.
    		Starting stats...Done.
    After restart, all services become unavailable and /var/log/zimbra.log fills with errors indicating SSL key issues. Nothing in the log files stands out to me as smoking gun... mostly smells to me like an issue with the deployment of the SSL key pair.

    Even though the services appear down, a zmcontrol status results in the following output:

    Code:
    zimbra@mail:~$ zmcontrol status
    Unable to determine enabled services from ldap.
    Enabled services read from cache. Service list may be inaccurate.
    Host mail.xxxxxxx.xxx
    	antispam                Running
    	antivirus               Running
    	convertd                Running
    	ldap                    Running
    	logger                  Running
    	mailbox                 Running
    	memcached               Running
    	mta                     Running
    	snmp                    Running
    	spell                   Running
    	stats                   Running
    zimbra@mail:~$
    I've performed several clean installs of ZCS, and each ends with the same issue. I've attached the console log from my ZCS install and cert deployment, the zimbra.log, and the zimbra installer log. All logs from this server are available for further review, and I'm willing to try just about anything to resolve this issue.

    Thanks!

    Robert
    Attached Files Attached Files

  2. #2
    y@w's Avatar
    y@w
    y@w is offline Moderator
    Join Date
    Jan 2008
    Posts
    658
    Rep Power
    8

    Default

    Oct 16 02:58:16 mail zimbramon[30915]: 30915:info: zmmtaconfig: Skipping getAllMtaAuthURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.net.ssl.SSLHandshakeException sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderE xception: unable to find valid certification path to requested target)
    Have a look at the troubleshooting section of this page: Installing a Verisign Test Certificate - Zimbra :: Wiki

    Looks like you're hitting the same error. That may help, or at least get you in the right direction..

  3. #3
    rwc101010 is offline Active Member
    Join Date
    Dec 2006
    Location
    SF Bay Area
    Posts
    30
    Rep Power
    8

    Default

    Quote Originally Posted by y@w View Post
    Have a look at the troubleshooting section of this page: Installing a Verisign Test Certificate - Zimbra :: Wiki

    Looks like you're hitting the same error. That may help, or at least get you in the right direction..
    No, that article didn't come up in any of my searching. So, the gist of the wiki page is that the java keystore lacks the proper CA chain to validate the cert?

    Looking at the wiki page, it appears the following command performs the import:

    Code:
    # /opt/zimbra/java/bin/keytool -import -alias (ALIAS) -keystore /opt/zimbra/java/jre/lib/security/cacerts -storepass (PASSWORD) -file /opt/zimbra/conf/ca/commercial_ca.pem
    My only question would be what value is substituted for (ALIAS) in the above command, and where I find the cacerts (PASSWORD).

    Thanks for the tip!
    Last edited by rwc101010; 10-15-2009 at 10:02 PM. Reason: Changed brackets around ALIAS and PASSWORD for clarity and rendering.

  4. #4
    rwc101010 is offline Active Member
    Join Date
    Dec 2006
    Location
    SF Bay Area
    Posts
    30
    Rep Power
    8

    Default And we have a winner!!!

    Thanks to y@w for suggesting the root java keystore.

    Running the following command as root, and restarting ZCS resolved the issue:

    Code:
    /opt/zimbra/java/bin/keytool -import -alias root -keystore /opt/zimbra/java/jre/lib/security/cacerts -storepass changeit -file /opt/zimbra/conf/ca/commercial_ca.pem
    My questions is, why would a valid cert chain validate as good when using the zmcertmgr verifycrt comm command, but fail to validate using the java root keychain. Seems like a pretty big disconnect to have two inconsistent CA root keychains in the same application stack....

    Thanks again for the pointer!

    Robert
    Last edited by rwc101010; 10-15-2009 at 11:06 PM. Reason: Added note that the command should be run as root

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Trouble Sending mail - All Messages deferred!
    By SiteDiscovery in forum Administrators
    Replies: 7
    Last Post: 09-03-2009, 04:52 AM
  2. Moving ZCS from 32bit to 64bit or vice versa ?
    By mark_orion in forum Administrators
    Replies: 1
    Last Post: 08-26-2008, 05:30 AM
  3. SSL Cert Questions
    By playnada in forum Administrators
    Replies: 3
    Last Post: 05-06-2008, 10:22 AM
  4. Commercial Cert for Zimbra Web
    By mwyant in forum Installation
    Replies: 4
    Last Post: 07-17-2007, 10:22 AM
  5. ZCS 3.2 Beta Available
    By KevinH in forum Announcements
    Replies: 31
    Last Post: 07-07-2006, 03:46 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •