4.5 to 5.0 OSS upgrade on Ubuntu kills all LDAP data

[RoadRunnr]

Hi,

I have tried this 3 times in a test server running in VMware and whatever i tried, the upgrade to 5.0 always killed all data in the LDAP tree.

Looking through the install/upgrade script:

1. the LDAP tree is dumped into an LDIF (ldap.bak)
2. then the openldap-data directory is moved to openldap-data.BAK
3. the package installation then creates a new openldat-data directory.

But i can't find anything that restores the old data into the new ldap directory.

There is this telling snipped in the log output (hostname edited):

Restoring existing configuration file from /opt/zimbra/.saveconfig/config.save...done
Operations logged to /tmp/zmsetup.12292007-170725.log
Starting ldap...done.
ERROR: account.NO_SUCH_SERVER (no such server: zimbra.test)
ERROR: account.NO_SUCH_SERVER (no such server: zimbra.test)
Setting defaults...ERROR: account.NO_SUCH_SERVER (no such server: zimbra.test)

This happens because the LDAP tree is empty at this point and i guess it should not be.

The installation later then add some ham and spam users. And those are the only ones present in the LDAP after the upgrade.openldap-data.BAK

BTW: moving openldap-data to openldap-data.BAK is another problem. The existens of openldap-data.BAK is not tested before and the install script will happily move openldap-data as a subdirectory into a exisiting openldap-data.BAK directory.

Andreas

[jholder]

The only way that would happen is if the installer was repeatedly run after an installation failure.

Once it fails the first time, the localconfig.xml is blank. When you run the second time, it sees the blank localconfig and thinks it's a new install.

Please provide your /opt/zimbra/.install_history and /tmp/zmsetup.log and ./tmp/install.log.xxxx

[RoadRunnr]

Hi jholder,

Logs attached.

All attempts used a working zimbra 4.5.6 install as basis. localconfig.xml is not blank.

I've tried to follow the install script, and to me it appears as if renaming the working ldap data directory to .BAK will occure every time, regardless of the state of localconfig.xml.
=> removeExistingInstall is called directly before "Installing packages" and it will move openldap-data to openldap-data.BAK and no one will ever read the data from that directory anymore.

[fireman949]

Originally Posted by jholder

The only way that would happen is if the installer was repeatedly run after an installation failure.

Once it fails the first time, the localconfig.xml is blank. When you run the second time, it sees the blank localconfig and thinks it's a new install.

So what happens when you make a snapshot LVM backup (tar.gz) and when the install fails the first (second, third in my case) time you rm -Rf everything in /opt/zimbra and start over by un-tar'ing the backup??

I'm having the exact same issue as the previous poster described. I'll check and gather some logs, but I can confirm that I didn't repeatedly run the installer after an installation failure without completely restoring from backup (unless the installer is looking somewhere outside /opt/zimbra for localconfig.xml)

[jholder]

Roadrunner,
Please provide evidence that #2 is happening. It shouldn't move the directory, but create it from the backup.

If you guys think this is a bug, please feel free to file it.
It looks to me like the installer was repeatedly run 6 times.

As I mentioned, this can sometimes cause an issue with localconfig.

To test, please cat the file and post the results. I mis-spoke earlier. It's not blank, but is incomplete.

[fireman949]

I was able to complete a successful upgrade from 4.5.x to 5.0GA.

Here is what I did differently from the 1st two times:

I had iptables running and I change it to limit iptables to just block port 25 (so that incoming e-mail is differed to secondary MX).

Then.. at one point in this text it hung. I was executing ls -latr on /var/tmp and saw a file zmcontrol.out.dXeXI that was new.. I tail'd that file and saw "password : " like when sudo is executed.

I remembered from my upgrade (and then downgrade) that 5.0 makes modifications to /etc/sudoers that are not compatible with 4.5.x so until I found the solution I added a rule that was :
%zimbra ALL=NOPASSWD: ALL (I know.. not good but it did the job for the time being)

When I saw the prompt for a password in that log/out file I quickly added that line to /etc/sudoers. It was somewhere in the install where I saw this that it hung:

Restoring existing configuration file from /opt/zimbra/.saveconfig/config.save...done
Operations logged to /tmp/zmsetup.12292007-173217.log
Starting ldap...done.
Setting defaults...done.
Upgrading from 4.5.10_GA_1575 to 5.0.0_GA_1869
Stopping zimbra services

(here I think)

Verifying /opt/zimbra/conf/my.cnf
Starting mysql
Starting logger mysql


I was able to finish the upgrade without error. I don't know which of the two changes fixed the problem, but that did the job.

If Zimbra staff needs logs, I can provide, but I suspect the iptables rules were the issue.

-Eric

[fireman949]

To further clarify..

You (roadrunnr) will need a pristine backup of your original install before you started attempting to upgrade. Every time I attempted upgrade, I started by removing everything in /opt/zimbra and restoring from backup.

[RoadRunnr]

to fireman,

Originally Posted by fireman949

To further clarify..

You (roadrunnr) will need a pristine backup of your original install before you started attempting to upgrade. Every time I attempted upgrade, I started by removing everything in /opt/zimbra and restoring from backup.

My test system is running in a Vmware and i took a VMware snapshot of the working 4.5.6 installation. Everytime i try, i just reset the image to that snapshot and start from there.

Also, i don't see any issue with the /etc/sudoers file. This will IMHO only happen if you upgrade to 5.0, delete that instance and restore a backup that has only /opt/zimbra. This will leave the now newer /etc/sudoers in place, causing the password prompt.

The main difference i see from your attempt is that you upgrade from 4.5.10 to 5.0 while i have 4.5.6 instance to start with

[RoadRunnr]

Hi,

ok, i found the problem. My dpkg database did not correctly reflect the state of system. This together with what i thing is quite a questionable test caused the moving of the openldap-data directory.

The install script will move openldap-data to openldap-data.BAK if /opt/zimbra/openldap/var/openldap-data/ exists. This directory normaly can not exist, as the zimbra-ldap package has been removed only a few lines earlier.

However, /opt/zimbra/openldap is a symlink to the ldap instance installed by zimbra-ldap. If this symlink points somewhere else, then the script will move the data directory, thereby removing all ldap data.

Andreas