[SOLVED] DB Version Mismatch after upgrade from RC1 to RC2

[xflip]

Hi!
I've updated from Zimbra 5.0 RC1 to RC2 today and now I can't use the webinterfaces (user & admin) or IMAP anymore.

Trying to log into the webmailer fails with

A network service error has occurred (zclient/io).

A look at /opt/zimbra/log/mailbox.log tells me, that the installer didn't updated the database correctly. I tried to run the installer another time, but nothing changed. Is there anything I can do about that?

Thanks!

2007-11-22 09:54:07,220 ERROR [main] [] Versions - DB Version Mismatch: ours="48" from DB="47"
2007-11-22 09:54:07,221 FATAL [main] [] soap - Unable to start servlet
java.lang.RuntimeException: Data version mismatch.  Reinitialize or upgrade the backend data store.
        at com.zimbra.cs.util.Zimbra.startup(Zimbra.java:89)
        at com.zimbra.soap.SoapServlet.init(SoapServlet.java:114)
        at javax.servlet.GenericServlet.init(GenericServlet.java:241)
        at org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:433)
        at org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:256)

[phoenix]

Which operating system? Which version of Zimbra OSS or NE? Single server or multi server? Did you take a backup before the upgrade? If you did I'd suggest restoring to a working version.

[xflip]

Hi phoenix,

Debian 4.0
Zimbra OSS on a single server.
Yes, I did a backup, but I really want to do an upgrade from RC1 to RC2 and I don't see what should change if I restore my backup and reupgrade again. Or do you think that might really work?

bye

[phoenix]

What does 'zmcontrol -v' show when you run it as the zimbra user?

PS You could also see what's in the file /opt/zimbra/db/data/(server_fqdn).err

[xflip]

zimbra@zimbra:/root/zcs-5.0.0_RC2$ zmcontrol -v


Release 5.0.0_RC2_1745.DEBIAN4.0 DEBIAN4.0

... and the mailbox server seems running.

zimbra@zimbra:/root/zcs-5.0.0_RC2$ zmcontrol status
Host zimbra.example.de
        antispam                Running
        antivirus               Running
        ldap                    Running
        logger                  Running
        mailbox                 Running
        mta                     Running
        stats                   Running

/opt/zimbra/db/data/(server_fqdn).err:

071122 10:42:28  mysqld started
071122 10:42:30  InnoDB: Started; log sequence number 0 43553723
071122 10:42:30 [Note] /opt/zimbra/mysql/libexec/mysqld: ready for connections.
Version: '5.0.45-log'  socket: '/opt/zimbra/db/mysql.sock'  port: 7306  Source distribution
071122 11:06:26 [Note] /opt/zimbra/mysql/libexec/mysqld: Normal shutdown

071122 11:06:26  InnoDB: Starting shutdown...
071122 11:06:29  InnoDB: Shutdown completed; log sequence number 0 43553733
071122 11:06:29 [Note] /opt/zimbra/mysql/libexec/mysqld: Shutdown complete

071122 11:06:29  mysqld ended

071122 11:18:16  mysqld started
071122 11:18:18  InnoDB: Started; log sequence number 0 43553733
071122 11:18:18 [Note] /opt/zimbra/mysql/libexec/mysqld: ready for connections.
Version: '5.0.45-log'  socket: '/opt/zimbra/db/mysql.sock'  port: 7306  Source distribution

So the database server itself looks quite normal. Any ideas what I can do now?

[Dirk]

I also have a server running debian, the rc2 upgrade is currently downloading and once I have the backup completed, I'll try the upgrade and see what happens.

I hope not to get the same problem, but if I do then it's another set of eyes looking at the problem

[xflip]

I just looked into the logs of my first install and found this:

...
Starting ldap...done.
WARNING: Unknown package installed for store.
Setting defaults...done.
Setting defaults from existing config...done.
...

This didn't occur in the logs of the second try.

I shut down Zimbra manually in the frist run and let install.sh do the shut down in the second try. Maybe this might help.

[xflip]

I just removed RC2, restored my backup (RC1), reinstalled RC2 but the database version conflict persists.

[dijichi2]

There seems to be a nasty bug in the post-install setup which prevents vital upgrade actions, thus resulting in this db mismatch. I had the same problem when using rc1-debian4 package to upgrade so went back to the ubuntu6 hack. however this hack didn't work for me for rc2 so I was forced to find a workaround. I think this issue is restricted to the Debian4 build, I haven't seen it elsewhere.

This warning:
WARNING: Unknown package installed for store.
is an odd one, and comes from getInstalledPackages{} and seems to come because it can't correctly match up the store entry from:
zmprov gs |grep zimbraServiceInstalled

This then prevents the actual upgrade actions from happening, even though the logs partially claim otherwise. The workaround is either to fix this root cause (dunno how, didn't have time enough to work that out), or you can probably bodge correct zimbraServiceInstalled entries with zmprov -l somehow, but here is the workaround I used:

1. Make sure you have the correct packages installed for the /opt/zimbra present - if you had to restore /opt/zimbra from tar/rsync backup, install the corresponding packages with install.sh -s first (hope you still have the RC1 tarball somewhere, doesn't seem to exist on zimbra download site - they didn't seem to think we might need archives).

2. Start the RC2 installer with -s flag (this just installs the packages, doesn't start the postinstall setup):
./install.sh -s

3. When the installer is complete, then we need to do the upgrades manually before the installer starts (otherwise it goes wrong):
su - zimbra
mysql.server start
cd /opt/zimbra/libexec/scripts
./migrate20070928-ScheduledTaskIndex.pl
mysql.server stop
zmlocalconfig -e zimbra_mailbox_purgeable=true
zmlocalconfig tomcat_thread_stack_size
- if set, use this value in next step, otherwise use 256k
zmlocalconfig -e mailboxd_thread_stack_size=256k
- if it was set: zmlocalconfig -u tomcat_thread_stack_size
exit (back to root user)

4. First, open a second window/shell/ssh to the server. OK, now run the setup from the first window:
/opt/zimbra/libexec/zmsetup.pl
- By the time mailboxd starts up towards the end of the setup, one more upgrade must be completed. The setup starts and stops ldap service several times, you need to run this command when the ldap service is up (you might need to run it a couple of times before its succesful, to catch the ldap server when its running):
zmprov -l mcf zimbraHttpNumThreads 100

And hey presto, you should have a running RC2 on Debian 4, using the Debian 4 binary, something I've been looking forward to for some time now!

** UPDATE - See later posting in this thread for root cause and fix, particularly if you're upgrading from before 5.0_RC1

[jholder]

Great job!

Sticking thread.

Please file a bug on this issue.