Zimbra

jterhune · #1 (**permalink**) 08-11-2009, 05:01 PM

I ran the update from 5.0.11 with no major issues (some corrupted tables that I had to repair for zimbra logger). The install finished, but hung at starting services. After waiting for quite awhile, I rebooted.

Everything appeared to come up properly, but the server is not working. Here are the oddities I have found so far:

zmcontrol status yields "Unable to determine enabled services from ldap. Enabled services read from cache. Service list may be inaccurate." and reports everything running.

ps aux | grep amavisd yields quite a few "amavisd (virgin child)" I don't believe this is how things should be.

I've done a chown zimbra:zimbra and a fixperms already, but we still aren't up and running.

Any advice?

phoenix · #2 (**permalink**) 08-11-2009, 11:28 PM

Shutdown Zimbra the kill any remaining processes and restart Zimbra, see what errors are in the log files, you may also want to search the forums for the above error messages that you've posted.

jterhune · #3 (**permalink**) 08-12-2009, 09:05 AM

I've tried pretty much everything I've found on those errors to no avail.

I tried to re-run the upgrade, but it failed fairly dramatically this time. It looked like the installer still believed we were on 5.0.11 and it failed with an ldap broken pipe.

Here are some things that don't look right in logs:

clamd.log

Code:

LibClamAV Warning: ******************************************************
LibClamAV Warning: ***      Virus database timestamp in the future!   ***
LibClamAV Warning: ***  Please check the timezone and clock settings  ***
LibClamAV Warning: ******************************************************
LibClamAV Warning: ***********************************************************
LibClamAV Warning: ***  This version of the ClamAV engine is outdated.     ***
LibClamAV Warning: *** DON'T PANIC! Read http://www.clamav.net/support/faq ***
LibClamAV Warning: ***********************************************************
Tue Aug 11 03:38:41 2009 -> Loaded 608499 signatures.
Tue Aug 11 03:38:42 2009 -> TCP: Bound to port 3310
Tue Aug 11 03:38:42 2009 -> TCP: Setting connection queue length to 15
Tue Aug 11 03:38:42 2009 -> Limits: Global size limit set to 10240000 bytes.
Tue Aug 11 03:38:42 2009 -> Limits: File size limit set to 10240000 bytes.
Tue Aug 11 03:38:42 2009 -> Limits: Recursion level limit set to 16.
Tue Aug 11 03:38:42 2009 -> Limits: Files limit set to 10000.
Tue Aug 11 03:38:42 2009 -> Archive support enabled.
Tue Aug 11 03:38:42 2009 -> Archive: Blocking encrypted archives.
Tue Aug 11 03:38:42 2009 -> Algorithmic detection enabled.
Tue Aug 11 03:38:42 2009 -> Portable Executable support enabled.
Tue Aug 11 03:38:42 2009 -> ELF support enabled.
Tue Aug 11 03:38:42 2009 -> Mail files support enabled.
Tue Aug 11 03:38:42 2009 -> OLE2 support enabled.
Tue Aug 11 03:38:42 2009 -> PDF support enabled.
Tue Aug 11 03:38:42 2009 -> HTML support enabled.
Tue Aug 11 03:38:42 2009 -> Self checking every 600 seconds.

freshclam.log

Code:

--------------------------------------
freshclam daemon 0.95.1-broken-compiler (OS: linux-gnu, ARCH: i386, CPU: i686)
ClamAV update process started at Tue Aug 11 03:38:00 2009
WARNING: Your ClamAV installation is OUTDATED!
WARNING: Local version: 0.95.1 Recommended version: 0.95.2
DON'T PANIC! Read http://www.clamav.net/support/faq
main.cld is up to date (version: 51, sigs: 545035, f-level: 42, builder: sven)
Downloading daily-9680.cdiff [100%]
Downloading daily-9681.cdiff [100%]
Downloading daily-9682.cdiff [100%]
Downloading daily-9683.cdiff [100%]
Downloading daily-9684.cdiff [100%]
daily.cld updated (version: 9684, sigs: 64237, f-level: 43, builder: ccordes)
WARNING: Your ClamAV installation is OUTDATED!
WARNING: Current functionality level = 42, recommended = 43
DON'T PANIC! Read http://www.clamav.net/support/faq
Database updated (609272 signatures) from db.us.clamav.net (IP: 168.143.19.95)
WARNING: Clamd was NOT notified: Can't connect to clamd on localhost:3310
--------------------------------------

mailbox.log - no entries since update

spamtraining.log

Code:

Learned tokens from 0 message(s) (0 message(s) examined)
netset: cannot include 127.0.0.0/8 as it has already been included
bayes: synced databases from journal in 6 seconds: 1366 unique entries (2719 total entries)
expired old bayes database entries in 19 seconds
186505 entries kept, 70790 deleted
token frequency: 1-occurrence tokens: 71.48%
token frequency: less than 8 occurrences: 16.63%
20090810230039 Finished spamassassin training.
20090810234502 Starting spam/ham cleanup
Exception in thread "main" com.zimbra.common.service.ServiceException: system failure: Unable to connect to LDAP directory
ExceptionId:main:1249973103273:0aefee60fb2cb614
Code:service.FAILURE
	at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:253)
	at com.zimbra.cs.util.SpamExtract.getAccount(SpamExtract.java:466)
	at com.zimbra.cs.util.SpamExtract.main(SpamExtract.java:170)
Caused by: com.zimbra.common.service.ServiceException: system failure: ZimbraLdapContext
ExceptionId:main:1249973103273:0aefee60fb2cb614
Code:service.FAILURE
	at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:253)
	at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init>(ZimbraLdapContext.java:326)
	at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init>(ZimbraLdapContext.java:297)
	at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init>(ZimbraLdapContext.java:290)
	at com.zimbra.cs.account.ldap.LdapProvisioning.getConfig(LdapProvisioning.java:388)
	at com.zimbra.cs.util.SpamExtract.getAccount(SpamExtract.java:464)
	... 1 more
Caused by: javax.naming.CommunicationException: eagles.cui.edu:389 [Root exception is java.net.ConnectException: Connection refused]
	at com.sun.jndi.ldap.Connection.<init>(Connection.java:197)
	at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:118)
	at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1578)
	at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2627)
	at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:289)
	at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:175)
	at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:193)
	at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:136)
	at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:66)
	at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)
	at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:247)
	at javax.naming.InitialContext.init(InitialContext.java:223)
	at javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:134)
	at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init>(ZimbraLdapContext.java:312)
	... 5 more
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
	at java.net.Socket.connect(Socket.java:520)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at com.sun.jndi.ldap.Connection.createSocket(Connection.java:336)
	at com.sun.jndi.ldap.Connection.<init>(Connection.java:184)
	... 18 more
Exception in thread "main" com.zimbra.common.service.ServiceException: system failure: Unable to connect to LDAP directory
ExceptionId:main:1249973104142:fb3936d876e80dae
Code:service.FAILURE
	at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:253)
	at com.zimbra.cs.util.SpamExtract.getAccount(SpamExtract.java:466)
	at com.zimbra.cs.util.SpamExtract.main(SpamExtract.java:170)
Caused by: com.zimbra.common.service.ServiceException: system failure: ZimbraLdapContext
ExceptionId:main:1249973104142:fb3936d876e80dae
Code:service.FAILURE
	at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:253)
	at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init>(ZimbraLdapContext.java:326)
	at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init>(ZimbraLdapContext.java:297)
	at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init>(ZimbraLdapContext.java:290)
	at com.zimbra.cs.account.ldap.LdapProvisioning.getConfig(LdapProvisioning.java:388)
	at com.zimbra.cs.util.SpamExtract.getAccount(SpamExtract.java:464)
	... 1 more
Caused by: javax.naming.CommunicationException: eagles.cui.edu:389 [Root exception is java.net.ConnectException: Connection refused]
	at com.sun.jndi.ldap.Connection.<init>(Connection.java:197)
	at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:118)
	at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1578)
	at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2627)
	at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:289)
	at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:175)
	at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:193)
	at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:136)
	at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:66)
	at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)
	at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:247)
	at javax.naming.InitialContext.init(InitialContext.java:223)
	at javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:134)
	at com.zimbra.cs.account.ldap.ZimbraLdapContext.<init>(ZimbraLdapContext.java:312)
	... 5 more
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
	at java.net.Socket.connect(Socket.java:520)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at com.sun.jndi.ldap.Connection.createSocket(Connection.java:336)
	at com.sun.jndi.ldap.Connection.<init>(Connection.java:184)
	... 18 more
20090810234504 Finished spam/ham cleanup

zmmtaconfig.log

Code:

Tue Aug 11 03:43:13 2009  Skipping Global system configuration update.
Tue Aug 11 03:43:13 2009  gacf ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe) 
Tue Aug 11 03:43:14 2009  Skipping All Reverse Proxy URLs update.
Tue Aug 11 03:43:14 2009  Skipping getAllReverseProxyURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe) 
Tue Aug 11 03:43:16 2009  Skipping All Reverse Proxy Backends update.
Tue Aug 11 03:43:16 2009  Skipping getAllReverseProxyBackends ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe) 
Tue Aug 11 03:43:17 2009  Skipping All Memcached Servers update.
Tue Aug 11 03:43:17 2009  Skipping getAllMemcachedServers ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe) 
Tue Aug 11 03:43:18 2009  Skipping All MTA Authentication Target URLs update.
Tue Aug 11 03:43:18 2009  Skipping getAllMtaAuthURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe) 
Tue Aug 11 03:43:19 2009  Skipping Configuration for server eagles.cui.edu update.
Tue Aug 11 03:43:19 2009  gs:eagles.cui.edu ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: java.net.SocketException Broken pipe) 
Tue Aug 11 03:43:19 2009  Sleeping...Key lookup failed.

uxbod · #4 (**permalink**) 08-12-2009, 09:12 AM

Wiki :: Resetting ClamAV signature database

jterhune · #5 (**permalink**) 08-12-2009, 09:36 AM

ClamAV database moved, but...

Code:

zmprov ms `zmhostname` +zimbraServiceEnabled antivirus
ERROR: zclient.IO_ERROR (invoke Connection reset, server: localhost) (cause: java.net.SocketException Connection reset)

uxbod · #6 (**permalink**) 08-12-2009, 09:43 AM

Okay, so its not that

Code:

su - zimbra
zmcontrol -v
zmcontrol status
cat /etc/hosts
cat /etc/resolv.conf
dig yourdomain mx
dig yourdomain any
host `hostname` <- note backticks and not single quotes

When you did shutdown ZCS are you sure all Zimbra owner processes had died ?

jterhune · #7 (**permalink**) 08-12-2009, 09:49 AM

zmcontrol -v

Code:

Release 5.0.18_GA_3011.RHEL5_20090707164432 CentOS5

zmcontrol -status

Code:

Unable to determine enabled services from ldap.
Enabled services read from cache. Service list may be inaccurate.
Host eagles.cui.edu
        antispam                Running
        antivirus               Running
        ldap                    Running
        logger                  Running
        mailbox                 Running
        mta                     Running
        snmp                    Running
        spell                   Running
        stats                   Running

/etc/hosts

Code:

# Do not remove the following line, or various programs
# that require network functionality will fail.
172.16.1.92     eagles.cui.edu  eagles
127.0.0.1       localhost.localdomain   localhost
::1     localhost6.localdomain6 localhost6

/etc/resolv.conf

Code:

search cui.edu
nameserver 172.16.1.20
nameserver 172.16.1.24

dig yourdomain mx (note, we go through a spam gateway)

Code:

; <<>> DiG 9.3.3rc2 <<>> eagles.cui.edu mx
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46658
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; QUESTION SECTION:
;eagles.cui.edu.                        IN      MX

;; ANSWER SECTION:
eagles.cui.edu.         3600    IN      MX      1 cerberus.cui.edu.

;; ADDITIONAL SECTION:
cerberus.cui.edu.       3600    IN      A       172.16.0.252

;; Query time: 17 msec
;; SERVER: 172.16.1.20#53(172.16.1.20)
;; WHEN: Tue Aug 11 04:07:08 2009
;; MSG SIZE  rcvd: 73

dig yourdomain any

Code:

; <<>> DiG 9.3.3rc2 <<>> eagles.cui.edu any
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62415
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; QUESTION SECTION:
;eagles.cui.edu.                        IN      ANY

;; ANSWER SECTION:
eagles.cui.edu.         600     IN      A       172.16.1.151
eagles.cui.edu.         3600    IN      MX      1 cerberus.cui.edu.

;; ADDITIONAL SECTION:
cerberus.cui.edu.       3600    IN      A       172.16.0.252

;; Query time: 7 msec
;; SERVER: 172.16.1.20#53(172.16.1.20)
;; WHEN: Tue Aug 11 04:07:12 2009
;; MSG SIZE  rcvd: 89

host `hostname`

Code:

eagles.cui.edu has address 172.16.1.151
eagles.cui.edu mail is handled by 1 cerberus.cui.edu.
You have new mail in /var/spool/mail/zimbra

*edit* - to your last question, i always 'ps auxww | grep zimbra' to make sure they are all dead.

uxbod · #8 (**permalink**) 08-12-2009, 10:43 AM

So is a different MTA handling your inbound email then ?

jterhune · #9 (**permalink**) 08-12-2009, 10:45 AM

Our mx records point to our spam gateway, cerberus.cui.edu, which then passes mail to our zimbra instance, eagles.cui.edu.

None of these settings changed, though, everything was working fine before the attempted upgrade. Now, nothing

jterhune · #10 (**permalink**) 08-12-2009, 02:26 PM

Well, I have an old backup (like an idiot I didn't take one before I tried the update). Is it possible to mount the mailstore from the corrupted instance to the good one? If so, is it better to leave it as it is or to try and upgrade the backup copy to 5.0.18 first?

Zimbra

Downloads

Product Information

Toolbox

Why Join?