Updated live sync script

I have attached to this post some updates to the scripts on the Live Sync wiki page...
Server Live sync - Zimbra :: Wiki

I can't edit the Wiki page right now since the Wiki is undergoing maintenance and not accepting new edits.

The tgz file contains updated scripts plus a Nagios plugin and nrpe configuration. Note that on Ubuntu the plugin would be put in /usr/lib/nagios/plugins/contrib and under Redhat 64-bit it would be under /usr/lib64/nagios/plugins/contrib. Adjust the nrpe config accordingly.

Here is a paste of the changelog...

Version 2.0.0

1) Started changes log :)

2) Constants in upper case, variables in lower case

3) Exclude ldap log changes from inotify watch

4) Nagios plug-in to monitor time since last successful operation

5) Redologs are archived once successfully replayed

6) Redologs are retrieved from incremental backups on Zimbra Network edition

7) Redologs are purged independently after specified time on both servers

8) Magic path names and numbers now defined in constants

9) Log reporting by level and with filtering

10) Script aware of the java redolog replay process

11) convertd runs during redolog replay if enabled so indexing will work

Best regards,
Simon

Is this up on the Wiki (i.e. is the wiki no up to date)? Has anyone tried this script with 7.2.x and/or 8.0.x?

Thanks.

The Wiki is not up to date. I would like to update it but it is frozen for maintenance so I have to post updates here instead. I have some minor updates to the above script that I intend to post up tomorrow.

I know it works fine on 7.2. I haven't tested it on 8.0.x as yet.

A minor update added. Use file, live_sync_2.0.1.tar.gz.

Version 2.0.1

1) Preserve mailboxd runing state after sync to allow warmer standby and functioning HSM

2) Don't refuse to start because of stale pid files after a system reboot

I've made some updates for Zimbra 8. Please do not use earlier versions of this script on Zimbra 8 or it will get very stuck trying to sync the ldap directory.

Also for version 8, with regard to the old setting up instructions, the rsync command used to set up the standby server now has to be aware of sparse files. Otherwise you will end up with 80GB more disk space being used on the standby server compared to the live server.

Old command

Code:

---

rsync -aHz --force --delete live_server:/opt/zimbra/ /opt/zimbra/

---

New command

Code:

---

rsync -aHz --force --delete --sparse live_server:/opt/zimbra/ /opt/zimbra/

---

These are the new changes. Many thanks to those who provided feedback to improve the script.

1) Compatible with Zimbra version 8

2) Kill hanging sync_commands processes on live server

3) Rename LDAP_DIR constant to LDAP_TEMP_DIR

4) Make rsync of ldap database aware of sparse files

5) For Zimbra version 8 and above use ldif export/import instead of rsync

Oops. New version already. There is an egrep that won't work on RHEL systems that has been fixed in the sync_commands script that resulted in the purge time days not being passed.

The script is now aware of HSM. If use of HSM is detected then mailboxd is kept running on the standby server and sync operations are suspended while the HSM process is active on the standby server.

If anyone is interested. Here is the /etc/init.d script I use. I know it works on Ubuntu. It /should/ work on Redhat/CentOS too but needs verifying.

One the standby server I disable Zimbra on boot and enable this script on boot.

Tested it...Works for me on 7.2.2

First a huge thankyou to simonb for making this script available. This should really be incorporated into Zimbra, and they should pay him handsomely to do so!

Secondly, please can the Zimbra folk update the Live Sync wiki page to at least point to this forum thread. I wouldn't have found it unless simonb alerted me to it, and I wouldn't have found the version of the script that works on Zimbra 8.

OK. So I got this working on my installation and I had a couple of things which slowed me down, so I thought I'd mention them here in case they help anyone out. Again it would be nice to be able to put these in the wiki, but the page is locked.

1) Hostname. The hostname on live and failover servers should be the SAME. I felt that it should be, but had to check.

2) On the initial rsync, its OK to exclude the /opt/zimbra/backup directory. This is not needed. That saved me about half a terrabyte of bandwidth. You can use rsync's --exclude tag.

3) To check that the failover server is indeed copying over emails and documents as expected, its OK to stop the sync, and start up zimbra. ie
cd /opt/zimbra/live_sync
./live_syncd stop
zmcontrol start
(check Zimbra Email account / Admin panel, and then when happy ....)
zmcontrol stop
./live_syncd start
The sync process will figure out where it is and catch up again.

4) Watch out for file permissions! When I had synced over to the failover server, I tried to start zimbra (zmcontrol start) and got a message about 'zmmailboxctrl is not running'. Not good. In in /opt/zimbra/log/zmmailboxd.out I found some errors claiming that certain files weren't there, and that permission was denied on others. Finally I figured it out. On the old server, jetty was trying to run as UID 1003 and GID 1004. But on the mirror server: zimbra was gid=1001 and uid=1001

~$ id zimbra
uid=1001(zimbra) gid=1001(zimbra) groups=1001(zimbra),4(adm),5(tty),1002(postfix)

So to fix this I did the following on the new mirror server:
usermod -u 1003 zimbra
groupmod -g 1004 zimbra
find / -user 1001 -exec chown -h 1003 {} \;
find / -group 1001 -exec chgrp -h 1004 {} \;
(this takes a looong time)
usermod -g 1004 zimbra

And then zimbra would finally start!

So that's my feedback. In addition I can supply some metrics. On a server with 70 accounts, the server is syncing about 1Gb of data daily (less on weekends). So you might like to use that as a baseline for bandwidth.