mysql.server Not Running

[PS-Man]

Hi All,

I successfully upgraded from V5.0.09 to V5.0.10 and all was running. I then check my freshclam updates and was receiving error:
ERROR: Can't parse the config file /opt/zimbra/clamav-0.93.3/etc/clamd.conf

So I proceeded to correct this issue, and now freshclam is working; although I now get mysql.server not running in status. Starting mysql.server start returns no error although again does not start.

zimbra@mail:/root$ zmcontrol status
Host mail.concisecontrol.com.au
antispam Running
antivirus Running
imapproxy Running
ldap Running
logger Running
mailbox Stopped
mysql.server is not running
mta Running
snmp Running
spell Running
stats Stopped


Also the following:

zimbra@mail:/root$ zmmysqlstatus restart
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/opt/zimbra/db/mysql.sock' (2)
Error while running 'mysql --user=zimbra --password=REMOVED --database=zimbra --batch --skip-column-names'. at /opt/zimbra/bin/zmmysqlstatus line 243.

Any ideas?

Thanks.


Further info from mailbox.log:

2008-10-10 14:16:49,625 INFO [main] [] soap - Servlet SoapServlet starting up
2008-10-10 14:16:49,666 INFO [main] [] soap - Adding service AccountService to SoapServlet
2008-10-10 14:16:50,230 INFO [main] [] soap - Adding service MailService to SoapServlet
2008-10-10 14:16:50,420 INFO [main] [] soap - Adding service IMService to SoapServlet
2008-10-10 14:16:50,432 INFO [main] [] misc - version=5.0.10_GA_2638.UBUNTU6 release=20081003035628 builddate=20081003-0401 buildhost=build13.lab.zimbra.com
2008-10-10 14:16:50,433 INFO [main] [] misc - LANG environment is set to: en_AU.UTF-8
2008-10-10 14:16:50,434 INFO [main] [] misc - System property java.home=/opt/zimbra/jdk1.5.0_15/jre
2008-10-10 14:16:50,434 INFO [main] [] misc - System property java.runtime.version=1.5.0_15-b04
2008-10-10 14:16:50,434 INFO [main] [] misc - System property java.version=1.5.0_15
2008-10-10 14:16:50,434 INFO [main] [] misc - System property java.vm.info=mixed mode
2008-10-10 14:16:50,434 INFO [main] [] misc - System property java.vm.name=Java HotSpot(TM) Client VM
2008-10-10 14:16:50,434 INFO [main] [] misc - System property java.vm.version=1.5.0_15-b04
2008-10-10 14:16:50,434 INFO [main] [] misc - System property os.arch=i386
2008-10-10 14:16:50,434 INFO [main] [] misc - System property os.name=Linux
2008-10-10 14:16:50,434 INFO [main] [] misc - System property os.version=2.6.15-26-server
2008-10-10 14:16:50,434 INFO [main] [] misc - System property sun.arch.data.model=32
2008-10-10 14:16:50,434 INFO [main] [] misc - System property sun.cpu.endian=little
2008-10-10 14:16:50,435 INFO [main] [] misc - System property sun.cpu.isalist=
2008-10-10 14:16:50,435 INFO [main] [] misc - System property sun.os.patch.level=unknown
2008-10-10 14:16:50,482 INFO [main] [] system - Setting mysql connector property: maxActive=100
2008-10-10 14:16:50,483 INFO [main] [] system - Setting mysql connector property: maxActive=100
2008-10-10 14:16:54,737 WARN [main] [] misc - Could not establish a connection to the database. Retrying in 5 seconds.
com.zimbra.common.service.ServiceException: system failure: getting database connection
ExceptionId:main:1223608614730:44c5ae30442b5e3d
Code:service.FAILURE
at com.zimbra.common.service.ServiceException.FAILURE (ServiceException.java:253)
at com.zimbra.cs.db.DbPool.getConnection(DbPool.java: 236)
at com.zimbra.cs.util.Zimbra.waitForDatabase(Zimbra.j ava:122)
at com.zimbra.cs.util.Zimbra.startup(Zimbra.java:163)
at com.zimbra.cs.util.Zimbra.startup(Zimbra.java:137)
at com.zimbra.soap.SoapServlet.init(SoapServlet.java: 117)
at javax.servlet.GenericServlet.init(GenericServlet.j ava:241)
at org.mortbay.jetty.servlet.ServletHolder.initServle t(ServletHolder.java:433)
at org.mortbay.jetty.servlet.ServletHolder.doStart(Se rvletHolder.java:256)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.jetty.servlet.ServletHandler.initializ e(ServletHandler.java:612)
at org.mortbay.jetty.servlet.Context.startContext(Con text.java:139)
at org.mortbay.jetty.webapp.WebAppContext.startContex t(WebAppContext.java:1222)
at org.mortbay.jetty.handler.ContextHandler.doStart(C ontextHandler.java:501)
at org.mortbay.jetty.webapp.WebAppContext.doStart(Web AppContext.java:449)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.jetty.handler.HandlerCollection.doStar t(HandlerCollection.java:147)
at org.mortbay.jetty.handler.ContextHandlerCollection .doStart(ContextHandlerCollection.java:161)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.jetty.handler.HandlerCollection.doStar t(HandlerCollection.java:147)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.jetty.handler.HandlerWrapper.doStart(H andlerWrapper.java:117)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.jetty.handler.HandlerWrapper.doStart(H andlerWrapper.java:117)
at org.mortbay.jetty.Server.doStart(Server.java:217)
at org.mortbay.setuid.SetUIDServer.doStart(SetUIDServ er.java:100)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.xml.XmlConfiguration.main(XmlConfigura tion.java:979)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.mortbay.start.Main.invokeMain(Main.java:183)
at org.mortbay.start.Main.start(Main.java:497)
at org.mortbay.start.Main.main(Main.java:115)
Caused by: com.mysql.jdbc.exceptions.MySQLNonTransientConnect ionException: Server connection failure during transaction. Due to underlying exception: 'java.net.SocketException: java.net.ConnectException: Connection refused'.

** BEGIN NESTED EXCEPTION **

java.net.SocketException
MESSAGE: java.net.ConnectException: Connection refused

STACKTRACE:

java.net.SocketException: java.net.ConnectException: Connection refused
at com.mysql.jdbc.StandardSocketFactory.connect(Stand ardSocketFactory.java:156)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:276)
at com.mysql.jdbc.Connection.createNewIO(Connection.j ava:2769)
at com.mysql.jdbc.Connection.<init>(Connection.java:1 531)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonReg isteringDriver.java:266)
at java.sql.DriverManager.getConnection(DriverManager .java:525)
at java.sql.DriverManager.getConnection(DriverManager .java:140)
at org.apache.commons.dbcp.DriverManagerConnectionFac tory.createConnection(DriverManagerConnectionFacto ry.java:97)
at org.apache.commons.dbcp.PoolableConnectionFactory. makeObject(PoolableConnectionFactory.java:300)
at org.apache.commons.pool.impl.GenericObjectPool.bor rowObject(GenericObjectPool.java:816)
at org.apache.commons.dbcp.PoolingDataSource.getConne ction(PoolingDataSource.java:140)
at com.zimbra.cs.db.DbPool.getConnection(DbPool.java: 227)
at com.zimbra.cs.util.Zimbra.waitForDatabase(Zimbra.j ava:122)
at com.zimbra.cs.util.Zimbra.startup(Zimbra.java:163)
at com.zimbra.cs.util.Zimbra.startup(Zimbra.java:137)
at com.zimbra.soap.SoapServlet.init(SoapServlet.java: 117)
at javax.servlet.GenericServlet.init(GenericServlet.j ava:241)
at org.mortbay.jetty.servlet.ServletHolder.initServle t(ServletHolder.java:433)
at org.mortbay.jetty.servlet.ServletHolder.doStart(Se rvletHolder.java:256)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.jetty.servlet.ServletHandler.initializ e(ServletHandler.java:612)
at org.mortbay.jetty.servlet.Context.startContext(Con text.java:139)
at org.mortbay.jetty.webapp.WebAppContext.startContex t(WebAppContext.java:1222)
at org.mortbay.jetty.handler.ContextHandler.doStart(C ontextHandler.java:501)
at org.mortbay.jetty.webapp.WebAppContext.doStart(Web AppContext.java:449)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.jetty.handler.HandlerCollection.doStar t(HandlerCollection.java:147)
at org.mortbay.jetty.handler.ContextHandlerCollection .doStart(ContextHandlerCollection.java:161)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.jetty.handler.HandlerCollection.doStar t(HandlerCollection.java:147)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.jetty.handler.HandlerWrapper.doStart(H andlerWrapper.java:117)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.jetty.handler.HandlerWrapper.doStart(H andlerWrapper.java:117)
at org.mortbay.jetty.Server.doStart(Server.java:217)
at org.mortbay.setuid.SetUIDServer.doStart(SetUIDServ er.java:100)
at org.mortbay.component.AbstractLifeCycle.start(Abst ractLifeCycle.java:40)
at org.mortbay.xml.XmlConfiguration.main(XmlConfigura tion.java:979)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.mortbay.start.Main.invokeMain(Main.java:183)
at org.mortbay.start.Main.start(Main.java:497)
at org.mortbay.start.Main.main(Main.java:115)


** END NESTED EXCEPTION **


Attempted reconnect 3 times. Giving up.
at com.mysql.jdbc.SQLError.createSQLException(SQLErro r.java:888)
at com.mysql.jdbc.Connection.createNewIO(Connection.j ava:2838)
at com.mysql.jdbc.Connection.<init>(Connection.java:1 531)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonReg isteringDriver.java:266)
at java.sql.DriverManager.getConnection(DriverManager .java:525)
at java.sql.DriverManager.getConnection(DriverManager .java:140)
at org.apache.commons.dbcp.DriverManagerConnectionFac tory.createConnection(DriverManagerConnectionFacto ry.java:97)
at org.apache.commons.dbcp.PoolableConnectionFactory. makeObject(PoolableConnectionFactory.java:300)
at org.apache.commons.pool.impl.GenericObjectPool.bor rowObject(GenericObjectPool.java:816)
at org.apache.commons.dbcp.PoolingDataSource.getConne ction(PoolingDataSource.java:140)
at com.zimbra.cs.db.DbPool.getConnection(DbPool.java: 227)
... 33 more
2008-10-10 14:17:03,774 WARN [main] [] misc - Could not establish a connection to the database. Retrying in 5 seconds.
com.zimbra.common.service.ServiceException: system failure: getting database connection
ExceptionId:main:1223608623774:44c5ae30442b5e3d

[mmorse]

Contents of /opt/zimbra/db/data/hostname.err ?

[PS-Man]

/opt/zimbra/db/ directory is empty! No hostname.err to locate.

[PS-Man]

So after upgrading to V5.0.10 and subsequently fixing the Freshclam update error I did not log out correctly of the "su Zimbra" console. So when I issued the command:

[root@mail:/opt/zimbra/clamav]# rm -rf db/

to create a fresh clamav DB - I deleted the /opt/zimbra/DB contents.

Lucky I ran an Arconis image prior and my /db contents has been restored.

Although same issue mailbox stopped, mysql.server not running.

zimbra@mail:/root$ zmcontrol status
Host mail.concisecontrol.com.au
antispam Running
antivirus Running
imapproxy Running
ldap Running
logger Running
mailbox Stopped
mysql.server is not running
mta Running
snmp Running
spell Running
stats Stopped

...and no /opt/zimbra/db/data/hostname.err file.

When I try to connect to "mysql mysql" I get the following error:

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/opt/zimbra/db/mysql.sock' (2)

[PS-Man]

Issue has been resolved by Zimbra Support

[mlodick]

More information please? I'm seeing the same thing.

[PS-Man]

Zimbra support performed the following fixes. The basic issue was an out-of-sync sql database.

1. Modified the /opt/zimbra/my.cnf file to restore following values.
innodb_buffer_pool_size = 1024000000
innodb_flush_method = O_DIRECT

2. Manually set the db.version to 53 which was 52 earlier. May be you put an older directory /opt/zimbra/db/data

3. Also turned off imapproxy which is not necessary in your single server installation.

I'm sorry I cannot be more specific as this was all the info. provided.