Page 1 of 3 123 LastLast
Results 1 to 10 of 28

Thread: Updated live sync script

  1. #1
    simonb is offline Trained Alumni
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    9

    Default Updated live sync script

    I have attached to this post some updates to the scripts on the Live Sync wiki page...
    Server Live sync - Zimbra :: Wiki

    I can't edit the Wiki page right now since the Wiki is undergoing maintenance and not accepting new edits.

    The tgz file contains updated scripts plus a Nagios plugin and nrpe configuration. Note that on Ubuntu the plugin would be put in /usr/lib/nagios/plugins/contrib and under Redhat 64-bit it would be under /usr/lib64/nagios/plugins/contrib. Adjust the nrpe config accordingly.

    Here is a paste of the changelog...


    Version 2.0.0

    1) Started changes log

    2) Constants in upper case, variables in lower case

    3) Exclude ldap log changes from inotify watch

    4) Nagios plug-in to monitor time since last successful operation

    5) Redologs are archived once successfully replayed

    6) Redologs are retrieved from incremental backups on Zimbra Network edition

    7) Redologs are purged independently after specified time on both servers

    8) Magic path names and numbers now defined in constants

    9) Log reporting by level and with filtering

    10) Script aware of the java redolog replay process

    11) convertd runs during redolog replay if enabled so indexing will work

    Best regards,
    Simon
    Attached Files Attached Files

  2. #2
    n.sossonko is offline Active Member
    Join Date
    Sep 2011
    Location
    Miami
    Posts
    40
    Rep Power
    3

    Default

    Is this up on the Wiki (i.e. is the wiki no up to date)? Has anyone tried this script with 7.2.x and/or 8.0.x?

    Thanks.

  3. #3
    simonb is offline Trained Alumni
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    9

    Default

    The Wiki is not up to date. I would like to update it but it is frozen for maintenance so I have to post updates here instead. I have some minor updates to the above script that I intend to post up tomorrow.

    I know it works fine on 7.2. I haven't tested it on 8.0.x as yet.

  4. #4
    simonb is offline Trained Alumni
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    9

    Default

    A minor update added. Use file, live_sync_2.0.1.tar.gz.

    Version 2.0.1

    1) Preserve mailboxd runing state after sync to allow warmer standby and functioning HSM

    2) Don't refuse to start because of stale pid files after a system reboot
    Attached Files Attached Files

  5. #5
    simonb is offline Trained Alumni
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    9

    Default

    I've made some updates for Zimbra 8. Please do not use earlier versions of this script on Zimbra 8 or it will get very stuck trying to sync the ldap directory.

    Also for version 8, with regard to the old setting up instructions, the rsync command used to set up the standby server now has to be aware of sparse files. Otherwise you will end up with 80GB more disk space being used on the standby server compared to the live server.

    Old command
    Code:
    rsync -aHz --force --delete live_server:/opt/zimbra/ /opt/zimbra/
    New command
    Code:
    rsync -aHz --force --delete --sparse live_server:/opt/zimbra/ /opt/zimbra/
    These are the new changes. Many thanks to those who provided feedback to improve the script.

    1) Compatible with Zimbra version 8

    2) Kill hanging sync_commands processes on live server

    3) Rename LDAP_DIR constant to LDAP_TEMP_DIR

    4) Make rsync of ldap database aware of sparse files

    5) For Zimbra version 8 and above use ldif export/import instead of rsync
    Attached Files Attached Files
    Last edited by simonb; 02-11-2013 at 01:23 AM.

  6. #6
    simonb is offline Trained Alumni
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    9

    Default

    Oops. New version already. There is an egrep that won't work on RHEL systems that has been fixed in the sync_commands script that resulted in the purge time days not being passed.
    Attached Files Attached Files

  7. #7
    simonb is offline Trained Alumni
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    9

    Default

    The script is now aware of HSM. If use of HSM is detected then mailboxd is kept running on the standby server and sync operations are suspended while the HSM process is active on the standby server.
    Attached Files Attached Files

  8. #8
    simonb is offline Trained Alumni
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    9

    Default

    If anyone is interested. Here is the /etc/init.d script I use. I know it works on Ubuntu. It /should/ work on Redhat/CentOS too but needs verifying.

    One the standby server I disable Zimbra on boot and enable this script on boot.
    Attached Files Attached Files

  9. #9
    Raunaq's Avatar
    Raunaq is offline Zimbra Employee
    Join Date
    Nov 2012
    Location
    Bangalore
    Posts
    171
    Rep Power
    2

    Default

    Tested it...Works for me on 7.2.2

  10. #10
    plutocrat is offline Intermediate Member
    Join Date
    Dec 2009
    Posts
    19
    Rep Power
    5

    Default

    First a huge thankyou to simonb for making this script available. This should really be incorporated into Zimbra, and they should pay him handsomely to do so!

    Secondly, please can the Zimbra folk update the Live Sync wiki page to at least point to this forum thread. I wouldn't have found it unless simonb alerted me to it, and I wouldn't have found the version of the script that works on Zimbra 8.

    OK. So I got this working on my installation and I had a couple of things which slowed me down, so I thought I'd mention them here in case they help anyone out. Again it would be nice to be able to put these in the wiki, but the page is locked.

    1) Hostname. The hostname on live and failover servers should be the SAME. I felt that it should be, but had to check.

    2) On the initial rsync, its OK to exclude the /opt/zimbra/backup directory. This is not needed. That saved me about half a terrabyte of bandwidth. You can use rsync's --exclude tag.

    3) To check that the failover server is indeed copying over emails and documents as expected, its OK to stop the sync, and start up zimbra. ie
    cd /opt/zimbra/live_sync
    ./live_syncd stop
    zmcontrol start
    (check Zimbra Email account / Admin panel, and then when happy ....)
    zmcontrol stop
    ./live_syncd start
    The sync process will figure out where it is and catch up again.

    4) Watch out for file permissions! When I had synced over to the failover server, I tried to start zimbra (zmcontrol start) and got a message about 'zmmailboxctrl is not running'. Not good. In in /opt/zimbra/log/zmmailboxd.out I found some errors claiming that certain files weren't there, and that permission was denied on others. Finally I figured it out. On the old server, jetty was trying to run as UID 1003 and GID 1004. But on the mirror server: zimbra was gid=1001 and uid=1001

    ~$ id zimbra
    uid=1001(zimbra) gid=1001(zimbra) groups=1001(zimbra),4(adm),5(tty),1002(postfix)

    So to fix this I did the following on the new mirror server:
    usermod -u 1003 zimbra
    groupmod -g 1004 zimbra
    find / -user 1001 -exec chown -h 1003 {} \;
    find / -group 1001 -exec chgrp -h 1004 {} \;
    (this takes a looong time)
    usermod -g 1004 zimbra

    And then zimbra would finally start!

    So that's my feedback. In addition I can supply some metrics. On a server with 70 accounts, the server is syncing about 1Gb of data daily (less on weekends). So you might like to use that as a baseline for bandwidth.

Page 1 of 3 123 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Server Live sync
    By voxuantupc in forum Administrators
    Replies: 0
    Last Post: 02-29-2012, 01:32 AM
  2. Server Live Sync
    By mclain in forum Administrators
    Replies: 5
    Last Post: 01-23-2012, 11:12 AM
  3. How do you tell if antispam's being updated?
    By Baylink in forum Administrators
    Replies: 8
    Last Post: 04-09-2010, 05:30 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •