[SOLVED] Zimbra 5.0.10 to 5.0.11 upgrade hanging

[fizi]

I'm upgrading Zimbra 5.0.10 32bit to Zimbra 5.0.11 32bit on a CentOS 5 system.

It almost finishes the installation and appears to hang on a SQL query

Starting ldap...done.
Setting defaults...done.
Upgrading from 5.0.10_GA_2609 to 5.0.11_GA_2695
Stopping zimbra services
Verifying /opt/zimbra/conf/my.cnf
Starting mysql
Starting logger mysql
Tue Nov 18 20:46:07 2008: SELECT value FROM config WHERE name = 'db.version'
This appears to be 5.0.10_GA
Redolog Version: 1.23 New Redolog Version: 1.23
Stopping mysql
An upgrade of the logger schema is necessary from version 5
Running /opt/zimbra/libexec/scripts/migrateLogger6-qid.pl
Tue Nov 18 20:46:23 2008: SELECT value FROM config WHERE name = 'db.version'
Tue Nov 18 20:46:24 2008: Verified schema version 5.
Tue Nov 18 20:46:24 2008: Modifying postfix_qid size
Tue Nov 18 20:46:24 2008: alter table raw_logs modify postfix_qid VARCHAR(25);
alter table mta modify qid VARCHAR(25);

It almost looks like it shuts down MySQL and then triess to run a script on it.

I've done a CTRL-C out of the upgrade and re-run it and it hangs at the same point.

Any ideas?

[fizi]

Ah ha.

Am I correct in assuming that this version of Zimbra reformats all of the logs stored my MySQL?

 
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 9846 zimbra    25   0  692m 688m 2380 R  100 21.0   0:10.64 zmlogprocess
  602 zimbra    18   0  123m  24m 3652 S   64  0.7  71:49.35 mysqld

and that this may take a long while? (it's been about 1 hour now)

[phoenix]

Did you answer 'yes' to the database integrity request? If you did that may take a while to run.

[fizi]

I did answer 'yes'. I assumed it ran that integrity check before the upgrade occurred. Is that what it's doing now?

I should have posted all of the output I've gotten so far.

[root@tiny zcs-5.0.11_GA_2695.RHEL5.20081117051306]# ./install.sh --platform-override

Operations logged to /tmp/install.log.27695
Checking for existing installation...
    zimbra-ldap...FOUND zimbra-ldap-5.0.11_GA_2695
    zimbra-logger...FOUND zimbra-logger-5.0.11_GA_2695
    zimbra-mta...FOUND zimbra-mta-5.0.11_GA_2695
    zimbra-snmp...FOUND zimbra-snmp-5.0.11_GA_2695
    zimbra-store...FOUND zimbra-store-5.0.11_GA_2695
    zimbra-apache...FOUND zimbra-apache-5.0.11_GA_2695
    zimbra-spell...FOUND zimbra-spell-5.0.11_GA_2695
    zimbra-proxy...NOT FOUND
    zimbra-archiving...NOT FOUND
    zimbra-convertd...NOT FOUND
    zimbra-cluster...NOT FOUND
    zimbra-core...FOUND zimbra-core-5.0.11_GA_2695
ZCS upgrade from 5.0.11 to 5.0.11 will be performed.

Saving existing configuration file to /opt/zimbra/.saveconfig


PLEASE READ THIS AGREEMENT CAREFULLY BEFORE USING THE SOFTWARE.
ZIMBRA, INC. ("ZIMBRA") WILL ONLY LICENSE THIS SOFTWARE TO YOU IF YOU
FIRST ACCEPT THE TERMS OF THIS AGREEMENT. BY DOWNLOADING OR INSTALLING
THE SOFTWARE, OR USING THE PRODUCT, YOU ARE CONSENTING TO BE BOUND BY
THIS AGREEMENT. IF YOU DO NOT AGREE TO ALL OF THE TERMS OF THIS
AGREEMENT, THEN DO NOT DOWNLOAD, INSTALL OR USE THE PRODUCT.

License Terms for the Zimbra Collaboration Suite:
  http://www.zimbra.com/license/zimbra_public_eula_2.1.html


Press Return to continue

Checking for prerequisites...
     FOUND: NPTL
     FOUND: sudo-1.6.8p12-12
     FOUND: libidn-0.6.5-1.1
     FOUND: fetchmail-6.3.6-1.1
     FOUND: gmp-4.1.4-10
     FOUND: compat-libstdc++-296-2.96-138
     FOUND: compat-libstdc++-33-3.2.3-61
     FOUND: libtool-ltdl-1.5.22-6.1
     FOUND: /usr/lib/libstdc++.so.6
Checking for suggested prerequisites...
    FOUND: perl-5.8.8
Prerequisite check complete.
Checking current number of databases...

Do you want to verify message store database integrity? [Y] y
Verifying integrity of message store databases.  This may take a while.
Starting mysqld...done.
mysqld is alive
Generating report
No errors found

Checking for installable packages

Found zimbra-core
Found zimbra-ldap
Found zimbra-logger
Found zimbra-mta
Found zimbra-snmp
Found zimbra-store
Found zimbra-apache
Found zimbra-spell
Found zimbra-proxy


The Zimbra Collaboration Suite appears already to be installed.
It can be upgraded with no effect on existing accounts,
or the current installation can be completely removed prior
to installation for a clean install.

Do you wish to upgrade? [Y] y

Select the packages to install
    Upgrading zimbra-core
    Upgrading zimbra-ldap
    Upgrading zimbra-logger
    Upgrading zimbra-mta
    Upgrading zimbra-snmp
    Upgrading zimbra-store
    Upgrading zimbra-apache
    Upgrading zimbra-spell

Install zimbra-proxy [N]
Checking required space for zimbra-core
checking space for zimbra-store

Installing:
    zimbra-core
    zimbra-ldap
    zimbra-logger
    zimbra-mta
    zimbra-snmp
    zimbra-store
    zimbra-apache
    zimbra-spell

You appear to be installing packages on a platform different
than the platform for which they were built.

This platform is CentOS5
Packages found: RHEL5
This may or may not work.

Using packages for a platform in which they were not designed for
may result in an installation that is NOT usable. Your support
options may be limited if you choose to continue.


Install anyway? [N] y

The system will be modified.  Continue? [N] y

Shutting down zimbra mail
ERROR 2013 (HY000) at line 1: Lost connection to MySQL server during query

Backing up the ldap database...done.

Removing existing packages

   zimbra-ldap...done
   zimbra-logger...done
   zimbra-mta...done
   zimbra-snmp...done
   zimbra-store...done
   zimbra-spell...done
   zimbra-apache...done
   zimbra-core...done

Removing deployed webapp directories
Installing packages

    zimbra-core......zimbra-core-5.0.11_GA_2695.RHEL5-20081117051306.i386.rpm...done
    zimbra-ldap......zimbra-ldap-5.0.11_GA_2695.RHEL5-20081117051306.i386.rpm...done
    zimbra-logger......zimbra-logger-5.0.11_GA_2695.RHEL5-20081117051306.i386.rpm...done
    zimbra-mta......zimbra-mta-5.0.11_GA_2695.RHEL5-20081117051306.i386.rpm...done
    zimbra-snmp......zimbra-snmp-5.0.11_GA_2695.RHEL5-20081117051306.i386.rpm...done
    zimbra-store......zimbra-store-5.0.11_GA_2695.RHEL5-20081117051306.i386.rpm...done
    zimbra-apache......zimbra-apache-5.0.11_GA_2695.RHEL5-20081117051306.i386.rpm...done
    zimbra-spell......zimbra-spell-5.0.11_GA_2695.RHEL5-20081117051306.i386.rpm...done

Setting defaults from saved config in /opt/zimbra/.saveconfig/config.save
   HOSTNAME=XXXXXXXXXXXXXXXXXXXXXXX
   LDAPHOST=XXXXXXXXXXXXXXXXXXXXXXX
   LDAPPORT=389
   SNMPTRAPHOST=XXXXXXXXXXXXXXXXXXXXXXX
   SMTPSOURCE=XXXXXXXXXXXXXXXXXXXXXXX
   SMTPDEST=XXXXXXXXXXXXXXXXXXXXXXX
   SNMPNOTIFY=yes
   SMTPNOTIFY=yes
   LDAPROOTPW=XXXXXXXXXXXXXXXXXXXXXXX
   LDAPZIMBRAPW=XXXXXXXXXXXXXXXXXXXXXXX
   LDAPPOSTPW=XXXXXXXXXXXXXXXXXXXXXXX
   LDAPREPPW=XXXXXXXXXXXXXXXXXXXXXXX
   LDAPAMAVISPW=XXXXXXXXXXXXXXXXXXXXXXX
   LDAPNGINXPW=XXXXXXXXXXXXXXXXXXXXXXX
Restoring existing configuration file from /opt/zimbra/.saveconfig/config.save...done
Operations logged to /tmp/zmsetup.11182008-204512.log
Starting ldap...done.
Setting defaults...done.
Upgrading from 5.0.10_GA_2609 to 5.0.11_GA_2695
Stopping zimbra services
Verifying /opt/zimbra/conf/my.cnf
Starting mysql
Starting logger mysql
Tue Nov 18 20:46:07 2008: SELECT value FROM config WHERE name = 'db.version'
This appears to be 5.0.10_GA
Redolog Version: 1.23 New Redolog Version: 1.23
Stopping mysql
An upgrade of the logger schema is necessary from version 5
Running /opt/zimbra/libexec/scripts/migrateLogger6-qid.pl
Tue Nov 18 20:46:23 2008: SELECT value FROM config WHERE name = 'db.version'
Tue Nov 18 20:46:24 2008: Verified schema version 5.
Tue Nov 18 20:46:24 2008: Modifying postfix_qid size
Tue Nov 18 20:46:24 2008: alter table raw_logs modify postfix_qid VARCHAR(25);
alter table mta modify qid VARCHAR(25);

[jholder]

this is the logger mysql instance, and not the mailbox mysql instance.

There are a couple ways to get around this. The easiest is to uninstall the logger, and reinstall it later to get it running.

rpm -e `rpm -qa | grep zimbra-logger`
then rerun the installer

tell it to install the logger this time

You will loose all of your previous logger data, but it will get the system up.

If you're a NE customer, please contact support. Otherwise, this is probably the easiest path.

There was likely some corruption of somekind, which is a very known issue.

[fizi]

Just did what you described and I'm coming into the same road block.

[jholder]

that doesn't make sense.

do a rpm -qa | grep zimbra

and paste the output.

[fizi]

[root@tiny ~]# rpm -qa | grep zimbra
zimbra-spell-5.0.11_GA_2695.RHEL5-20081117051306
zimbra-snmp-5.0.11_GA_2695.RHEL5-20081117051306
zimbra-ldap-5.0.11_GA_2695.RHEL5-20081117051306
zimbra-store-5.0.11_GA_2695.RHEL5-20081117051306
zimbra-logger-5.0.11_GA_2695.RHEL5-20081117051306
zimbra-apache-5.0.11_GA_2695.RHEL5-20081117051306
zimbra-mta-5.0.11_GA_2695.RHEL5-20081117051306
zimbra-core-5.0.11_GA_2695.RHEL5-20081117051306

[fizi]

I did the following:

[root@tiny zcs-5.0.11_GA_2695.RHEL5.20081117051306]# rpm -qa |grep zimbra-logger
zimbra-logger-5.0.11_GA_2695.RHEL5-20081117051306
[root@tiny zcs-5.0.11_GA_2695.RHEL5.20081117051306]# ps aux |grep zimbra
root     11876  0.0  0.0   3912   652 pts/0    R+   03:01   0:00 grep zimbra
[root@tiny zcs-5.0.11_GA_2695.RHEL5.20081117051306]# rpm -e zimbra-logger-5.0.11_GA_2695.RHEL5-20081117051306

To resolve the problem and get my mail server working again I've simply removed the logger and not reinstalled it for now.

[fizi]

The root of this problem turned out to be a 4GB zimbra.log file in /var/log.

I deleted the log file and fixed my logrotate script and it resolved the problem.