[SOLVED] Zimbra 5.0.11 to 5.0.12 logger upgrade hanging?

[fizi]

I'm attempting to upgrade Zimbra 5.0.11 to 5.0.12 it looks like my upgrade is hanging.

I get this far:

Starting mysql
Starting logger mysql
This appears to be 5.0.11_GA
Redolog Version: 1.23 New Redolog Version: 1.23
Stopping mysql
An upgrade of the logger schema is necessary from version 5
Running /opt/zimbra/libexec/scripts/migrateLogger6-qid.pl
Wed Jan 28 13:13:49 2009: Verified schema version 5.
Wed Jan 28 13:13:49 2009: Modifying postfix_qid size

Is this script supposed to take a long time to complete? Originally it had been running for about 3 hours. I canceled it, renamed my /var/log/zimbra.log file and then retried the upgrade. So far same results.

I can run 'top' and I see Zimbra's MySQL using a lot of CPU so I'm assuming it's doing some work.

Should I leave it? or should I try the standard remove the logger, do the upgrade and then reinstall the logger?

I wasn't able to see any errors in the install log or the MySQL Error log.

I did a database integrity check before installing and all was good.

[fizi]

A little more digging shows the following:

/opt/zimbra/logger/db/data/zimbra_logger

-rw-rw---- 1 zimbra zimbra 3831882972 Jan 28 10:18 raw_logs.MYD
-rw-rw---- 1 zimbra zimbra 1275206656 Jan 28 10:18 raw_logs.MYI

Why on earth are these database tables 4GB!? My mail server does NOT see that much traffic in a day. I see maybe 100-200 e-mails a day.


** Update **
I got my upgrade to complete by renaming those two database tables to .old. Zimbra of course complains their missing now.

Is there anyway to reset the logger services? I'd like to remove and reinstall it and have it create a whole new database with blank tables.

[greenrenault]

Does it sound like this problem? Bug 33604 – Server Status Error (see Comment - Charles Gillet 2009-01-28 10:26:49)

This bug caused an upgrade from 5.0.11 to 5.0.12 to abort on open source edition. I had to go in and kill all the swatch and zmlogger processes before I could get it to go through cleanly. You should post a note in your support portal home page to watch out for this or patch your 5.0.12 installer to check for and kill these leftover processes.

[fizi]

Unfortunately no.

No Zimbra processes were running before the upgrade and the upgrade didn't produce any errors. It just sat there.

After getting things running again I checked my logger database tables and it appears that at least the 'raw_logs' table is corrupt. It's running a repair right now. I'm tempted to just truncate the damn table and be done with it.

[greenrenault]

Fair enough

[fizi]

So it looks like all of the database tables were corrupt. This could explain a few things.

I've run repairs on all of them except for 'zimbra_logger.raw_logs' which is still running (been a few hours). It's a 4GB table so I'll give it a little while longer.

The other table I'm having problems with is 'zimbra_logger.mta'. Here is the output I'm getting.

mysql> check table mta;
+-------------------+-------+----------+-------------------------------------------------------------------+
| Table             | Op    | Msg_type | Msg_text                                                          |
+-------------------+-------+----------+-------------------------------------------------------------------+
| zimbra_logger.mta | check | error    | Table upgrade required. Please do "REPAIR TABLE `mta`" to fix it! |
+-------------------+-------+----------+-------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> REPAIR TABLE `mta`;
+-------------------+--------+----------+--------------------------------------------------------------------------------------+
| Table             | Op     | Msg_type | Msg_text                                                                             |
+-------------------+--------+----------+--------------------------------------------------------------------------------------+
| zimbra_logger.mta | repair | Error    | Table './zimbra_logger/mta' is marked as crashed and last (automatic?) repair failed |
| zimbra_logger.mta | repair | Error    | Table 'mta' is marked as crashed and last (automatic?) repair failed                 |
| zimbra_logger.mta | repair | status   | Table is already up to date                                                          |
+-------------------+--------+----------+--------------------------------------------------------------------------------------+
3 rows in set (0.02 sec)

[jamturtle]

I have the same problem, did you manage to solve this. My upgrade is stuck...

This appears to be 5.0.10_GA
Redolog Version: 1.23 Ne Redolog Version: 1.23
Stopping mysql
An upgrade of the logger schema is necessary from version 5
Running /opt/zimbra/libexec/scripts/migrateLogger6-qid.pl
Wed Jan 28 13:13:49 2009: Verified schema version 5.
Wed Jan 28 13:13:49 2009: Modifying postfix_qid size

This has been going on for over an hour and I see the processor working but what is happening? Can I stop this at this point?
I have a fairly big /var/log/zimbra.log file could this affect it?
At this point I really do not care about the logs, I want the upgrade to finish or go back to 5.0.10 as long as I don't loose mail...

TIA

-James

[jamturtle]

ok just an update of what I did so others might benefit.

I stopped the script because it was stuck after waiting 2 hours I gave up. Then I went and killed all zimbra processes [yes stopping the script left a few things still doing something, (yes mainly logger stuff)].
Then I went to /var/log and renamed zimbra.log.
- then rpm -e zimbra-logger
I ran the installer again. This time it asked me if I wanted to install logger, so I said "Y"
The integrity check was good and the installer seemed to work fine this time.
I'm back in business!!

[fizi]

Great! I'm also up and running. I manually took out my zimbra.log and truncated my raw_logs table.

The only problem left is I can't repair the 'mta' table. I found a wiki article about backing up the entire database, recreating the database and then restoring the back. I'll give that a shot one of these days.

[vidave]

Always a bummer when ZCS upgrades die. Thanks for the posts. I had the same error, but also had to wipe out the old ZCS logger mysql database. I tried your fixes and got no quick results, just kept hanging. I went and looked and found my MySQL->logger.mta table was over 11 gig. No wonder I was having trouble.

NOT understanding Zimbra fully, I just figured "what the heck" and did the following:

mv /opt/zimbra/logger /opt/zimbra/logger-corrupt

rpm -e [installed zimbra logger rpm here]

killed off all zimbra stuff (after /etc/rc.d/init.d/zimbra stop)

Then restarted the install script. It FLEW through it -- apparently the portion of the install that deals with the logger is what really slows things down the most.

After nearly turning blue from holding my breath, I was relieved to see that after the install, all my accounts and email history were intact. Even better was that the admin GUI was all happy, and I don't have the error about the logger not working like I have for quite some time now.

Now....to get some sleep.