Results 1 to 4 of 4

Thread: Java won't connect to mysql, but all services are up. Server's been down all day!

  1. #1
    Gringo5000 is offline New Member
    Join Date
    Jan 2013
    Posts
    3
    Rep Power
    2

    Default Java won't connect to mysql, but all services are up. Server's been down all day!

    I'm in a mess. I tried updating 7.2.1 to 8.0.2 this morning and ldap blew up, so I tried to go back to 7.2.1 (without a real backup). 12 hours of massive problems later, I got install.sh to complete and all the services are running, but I can't connect to the services from the web or IMAP. The only errors I can find are in mailbox.log where java blows up saying it can't connect to the mysql db. I can connect to it fine from the command line.

    Running 7.2.1_GA_2790.RHEL6_64_20120815212147 UNKNOWN FOSS edition (on AWS). This is a single server install for a small organization and haven't been able to get email for most of a day. I would *really* appreciate some help.

    Code:
    Host mail.xxxx.xxx
            antispam                Running
            antivirus               Running
            ldap                    Running
            logger                  Running
            mailbox                 Running
            mta                     Running
            snmp                    Running
            spell                   Running
            stats                   Stopped
            zmconfigd               Running

    Here's what mailbox.log is spewing:
    Code:
    2013-01-04 08:30:08,742 WARN  [main] [] misc - Could not establish a connection to the database.  Retrying in 5 seconds.
    com.zimbra.common.service.ServiceException: system failure: getting database connection
    ExceptionId:main:1357288208742:b4537f7df005b7f6
    Code:service.FAILURE
            at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:253)
            at com.zimbra.cs.db.DbPool.getConnection(DbPool.java:314)
            at com.zimbra.cs.db.DbPool.getConnection(DbPool.java:278)
            at com.zimbra.cs.db.DbPool.waitForDatabase(DbPool.java:200)
            at com.zimbra.cs.db.DbPool.startup(DbPool.java:191)
            at com.zimbra.cs.util.Zimbra.startup(Zimbra.java:161)
            at com.zimbra.cs.util.Zimbra.startup(Zimbra.java:127)
            at com.zimbra.soap.SoapServlet.init(SoapServlet.java:130)
            at javax.servlet.GenericServlet.init(GenericServlet.java:241)
            at org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:440)
            at org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263)
            at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
            at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:685)
            at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
            at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1254)
            at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517)
            at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:471)
            at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
            at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
            at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
            at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
            at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
            at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
            at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
            at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
            at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
            at org.mortbay.jetty.handler.DebugHandler.doStart(DebugHandler.java:127)
            at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
            at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
            at org.mortbay.jetty.Server.doStart(Server.java:224)
            at org.mortbay.setuid.SetUIDServer.doStart(SetUIDServer.java:158)
            at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
            at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:985)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at org.mortbay.start.Main.invokeMain(Main.java:194)
            at org.mortbay.start.Main.start(Main.java:534)
            at org.mortbay.start.Main.start(Main.java:441)
            at org.mortbay.start.Main.main(Main.java:119)
    Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
            at sun.reflect.GeneratedConstructorAccessor15.newInstance(Unknown Source)
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
            at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
            at com.mysql.jdbc.Util.getInstance(Util.java:384)
            at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1015)
            at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
            at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:984)
            at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:929)
            at com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2226)
            at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2127)
            at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:774)
            at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:49)
            at sun.reflect.GeneratedConstructorAccessor13.newInstance(Unknown Source)
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
            at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
            at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:375)
            at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:289)
            at java.sql.DriverManager.getConnection(DriverManager.java:582)
            at java.sql.DriverManager.getConnection(DriverManager.java:154)
            at org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:78)
            at com.zimbra.cs.db.ZimbraConnectionFactory.createConnection(ZimbraConnectionFactory.java:64)
            at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
            at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1158)
            at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
            at com.zimbra.cs.db.DbPool.getConnection(DbPool.java:295)
            ... 39 more
    Caused by: java.sql.SQLException: Access denied for user 'zimbra'@'localhost' (using password: YES)
            at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1075)
            at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3566)
            at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3498)
            at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:919)
            at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:4004)
            at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1284)
            at com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2172)
    I have split DNS going, but it appears to work fine and the MX resolves appropriately.

    I've been scouring the forums for hours, but can't find a solution. One article was very close, but appears not to be the same problem.

  2. #2
    liverpoolfcfan's Avatar
    liverpoolfcfan is online now Outstanding Member
    Join Date
    Oct 2009
    Location
    Dublin, IRELAND
    Posts
    712
    Rep Power
    6

    Default

    This line
    Caused by: java.sql.SQLException: Access denied for user 'zimbra'@'localhost' (using password: YES)
    suggests that your zimbra user in the mysql user database may be corrupted.

    Try logging into the console
    su - zimbra
    mysql

    mysql> use database mysql;
    mysql> select * from user;

    and see if you have the correct "zimbra" user records in there.

    Do not paste the password here for obvious reasons.

    In my 7.2.1 database I have 3 zimbra user records - all with the same passwords - one each for "%", "localhost" and "localhost.localdomain"


    EDIT: Just searched for zimbra mysql password and found this wiki page
    http://wiki.zimbra.com/wiki/MySql_Password
    and this
    http://wiki.zimbra.com/wiki/Resettin...ySQL_Passwords

    They might be the best way to go about checking/changing.

    Good luck ...

  3. #3
    Gringo5000 is offline New Member
    Join Date
    Jan 2013
    Posts
    3
    Rep Power
    2

    Default

    Thank you very much for the response liverpoolfcfan. I can't see much of anything suspicious in the DB. Here's what I see:
    Code:
    mysql> select * from user;
    +-----------------------+--------+-------------------------------------------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+---------------+--------------+-----------+------------+-----------------+------------+------------+--------------+------------+-----------------------+------------------+--------------+-----------------+------------------+------------------+----------------+---------------------+--------------------+------------------+------------+--------------+----------+------------+-------------+--------------+---------------+-------------+-----------------+----------------------+
    | Host                  | User   | Password                                  | Select_priv | Insert_priv | Update_priv | Delete_priv | Create_priv | Drop_priv | Reload_priv | Shutdown_priv | Process_priv | File_priv | Grant_priv | References_priv | Index_priv | Alter_priv | Show_db_priv | Super_priv | Create_tmp_table_priv | Lock_tables_priv | Execute_priv | Repl_slave_priv | Repl_client_priv | Create_view_priv | Show_view_priv | Create_routine_priv | Alter_routine_priv | Create_user_priv | Event_priv | Trigger_priv | ssl_type | ssl_cipher | x509_issuer | x509_subject | max_questions | max_updates | max_connections | max_user_connections |
    +-----------------------+--------+-------------------------------------------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+---------------+--------------+-----------+------------+-----------------+------------+------------+--------------+------------+-----------------------+------------------+--------------+-----------------+------------------+------------------+----------------+---------------------+--------------------+------------------+------------+--------------+----------+------------+-------------+--------------+---------------+-------------+-----------------+----------------------+
    | localhost             | root   | xxxx | Y           | Y           | Y           | Y           | Y           | Y         | Y           | Y             | Y            | Y         | Y          | Y               | Y          | Y          | Y            | Y          | Y                     | Y                | Y            | Y               | Y                | Y                | Y              | Y                   | Y                  | Y                | Y          | Y            |          |            |             |              |             0 |           0 |               0 |                    0 |
    | mail.xxxx.xxx         | root   | xxxx | Y           | Y           | Y           | Y           | Y           | Y         | Y           | Y             | Y            | Y         | Y          | Y               | Y          | Y          | Y            | Y          | Y                     | Y                | Y            | Y               | Y                | Y                | Y              | Y                   | Y                  | Y                | Y          | Y            |          |            |             |              |             0 |           0 |               0 |                    0 |
    | 127.0.0.1             | root   | xxxx | Y           | Y           | Y           | Y           | Y           | Y         | Y           | Y             | Y            | Y         | Y          | Y               | Y          | Y          | Y            | Y          | Y                     | Y                | Y            | Y               | Y                | Y                | Y              | Y                   | Y                  | Y                | Y          | Y            |          |            |             |              |             0 |           0 |               0 |                    0 |
    | localhost             |        |                                           | N           | N           | N           | N           | N           | N         | N           | N             | N            | N         | N          | N               | N          | N          | N            | N          | N                     | N                | N            | N               | N                | N                | N              | N                   | N                  | N                | N          | N            |          |            |             |              |             0 |           0 |               0 |                    0 |
    | mail.xxxx.xxx         |        |                                           | N           | N           | N           | N           | N           | N         | N           | N             | N            | N         | N          | N               | N          | N          | N            | N          | N                     | N                | N            | N               | N                | N                | N              | N                   | N                  | N                | N          | N            |          |            |             |              |             0 |           0 |               0 |                    0 |
    | %                     | zimbra | xxxx | Y           | Y           | Y           | Y           | Y           | Y         | Y           | Y             | Y            | Y         | Y          | Y               | Y          | Y          | Y            | Y          | Y                     | Y                | Y            | Y               | Y                | Y                | Y              | Y                   | Y                  | Y                | Y          | Y            |          |            |             |              |             0 |           0 |               0 |                    0 |
    | localhost             | zimbra |xxxx | Y           | Y           | Y           | Y           | Y           | Y         | Y           | Y             | Y            | Y         | Y          | Y               | Y          | Y          | Y            | Y          | Y                     | Y                | Y            | Y               | Y                | Y                | Y              | Y                   | Y                  | Y                | Y          | Y            |          |            |             |              |             0 |           0 |               0 |                    0 |
    | localhost.localdomain | zimbra | xxxx | Y           | Y           | Y           | Y           | Y           | Y         | Y           | Y             | Y            | Y         | Y          | Y               | Y          | Y          | Y            | Y          | Y                     | Y                | Y            | Y               | Y                | Y                | Y              | Y                   | Y                  | Y                | Y          | Y            |          |            |             |              |             0 |           0 |               0 |                    0 |
    | localhost.localdomain | root   | xxxx | Y           | Y           | Y           | Y           | Y           | Y         | Y           | Y             | Y            | Y         | Y          | Y               | Y          | Y          | Y            | Y          | Y                     | Y                | Y            | Y               | Y                | Y                | Y              | Y                   | Y                  | Y                | Y          | Y            |          |            |             |              |             0 |           0 |               0 |                    0 |
    +-----------------------+--------+-------------------------------------------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+---------------+--------------+-----------+------------+-----------------+------------+------------+--------------+------------+-----------------------+------------------+--------------+-----------------+------------------+------------------+----------------+---------------------+--------------------+------------------+------------+--------------+----------+------------+-------------+--------------+---------------+-------------+-----------------+----------------------+
    9 rows in set (0.00 sec)
    Moving on to the password stuff, I'm now a bit nervous. First, it looks like some passwords might not be set right, so to make sure, I set out (via your links ) to set them. Here's localconfig:
    Code:
    [zimbra@mail ~]$ zmlocalconfig -s | grep mysql | grep password
    antispam_mysql_password = 
    antispam_mysql_root_password = 
    mysql_root_password = yyyyy
    zimbra_logger_mysql_password = 
    zimbra_mysql_password = XXXXX
    ...and this is where I see other issues:
    Code:
    [zimbra@mail ~]$ ldap status
    slapd running pid: 1563
    [zimbra@mail ~]$ zmldappasswd -r XXXX
    Updating local config and LDAP
    TLS: SSL connect attempt failed with unknown errorerror:0407006A:rsa routines:RSA_padding_check_PKCS1_type_1:block type is not 01
    [zimbra@mail ~]$ zmldappasswd YYYY       
    Updating local config and LDAP
    TLS: SSL connect attempt failed with unknown errorerror:0407006A:rsa routines:RSA_padding_check_PKCS1_type_1:block type is not 01
    [zimbra@mail ~]$ mysql.server status
    mysql is running with pid 8255
    [zimbra@mail ~]$ zmmypasswd --root QQQQQ
    /opt/zimbra/bin/mysqladmin -u root --password=zimbra password QQQQQ
    * Changed mysql root user password
    * Changed mysql root user password root@localhost
    [zimbra@mail ~]$ zmmypasswd  ZZZZZZ
    * Changed zimbra mysql user password
    [zimbra@mail ~]$ logmysql.server status
    -bash: logmysql.server: command not found
    [zimbra@mail ~]$ zmloggerctl status
    [zimbra@mail ~]$ ps ax | grep logger
     2758 ?        S      0:47 /usr/bin/perl /opt/zimbra/libexec/zmlogger
     3002 ?        S      0:00 zmlogger: zmrrdfetch: server
    20109 pts/0    S+     0:00 grep logger
    [zimbra@mail ~]$ zmmylogpasswd
    -bash: zmmylogpasswd: command not found
    First of all, you can see the TLS/SSL problem. I tried Googling that and following this article, I recreated my certs, but ended up getting the same failure. Any ideas? I'm not an openssl guy.

    Second is the logger stuff. zmcontrol shows it running, as does a ps, but I don't have the commands the web page references to set the password. Do you have those commands?

  4. #4
    Gringo5000 is offline New Member
    Join Date
    Jan 2013
    Posts
    3
    Rep Power
    2

    Default Fixed after restart

    It looks like things are back up and I'm receiving mail. What I didn't do after resetting the mysql passwords and recreating the certificate is restart the servers. After doing that and at the point of accepting the loss of lots email, things started working.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Replies: 2
    Last Post: 07-05-2012, 12:26 AM
  2. Admin panel services down but server's ones up
    By jcsnake in forum Administrators
    Replies: 2
    Last Post: 02-24-2012, 04:19 AM
  3. Connect to Zimbra's MySQL via ODBC?
    By jreinhart in forum Administrators
    Replies: 9
    Last Post: 04-13-2009, 08:58 PM
  4. [JAVA] as an admin connect to someone mailbox
    By chipmunk in forum Developers
    Replies: 1
    Last Post: 12-09-2008, 06:06 AM
  5. Replies: 0
    Last Post: 06-13-2008, 02:56 AM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •