Page 2 of 2 FirstFirst 12
Results 11 to 20 of 20

Thread: Zimbra OSE backup without stopping the services

  1. #11
    ewilen's Avatar
    ewilen is offline Moderator
    Join Date
    Jun 2008
    Location
    Berkeley, CA
    Posts
    1,474
    Rep Power
    9

    Default

    Wow. If you don't mind, what are your rsync command options? And how many individual files are in the source (roughly)?

  2. #12
    ewilen's Avatar
    ewilen is offline Moderator
    Join Date
    Jun 2008
    Location
    Berkeley, CA
    Posts
    1,474
    Rep Power
    9

    Default

    Also, I should ask what version of rsync you have installed. I've just read that version 3.0 introduces
    A new incremental-recursion algorithm is now used when rsync is talking to another 3.x version. This starts the transfer going more quickly (before all the files have been found), and requires much less memory.
    My RHEL came with 2.6.8 I believe; I'm going to try installing the newest version to see if that makes a difference.

  3. #13
    raj's Avatar
    raj
    raj is offline Moderator
    Join Date
    Oct 2005
    Location
    USA, Canada and India
    Posts
    777
    Rep Power
    10

    Default

    [root@zimbraXXX ~]# rsync --version
    rsync version 3.0.6 protocol version 30
    Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
    Web site: rsync
    Capabilities:
    64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, no symtimes
    rsync -avHK --timeout=999 -e ssh /opt/zimbra root@$BACKUP_SERVER_IP:/opt/$ZIMBRA_SERVER_NAME > 1.RSYNC.log
    rsync -avHK --delete --timeout=999 -e ssh /opt/zimbra root@$BACKUP_SERVER_IP:/opt/$ZIMBRA_SERVER_NAME > 2.RSYNC.log
    stop services
    rsync -avHK --delete --timeout=999 -e ssh /opt/zimbra root@$BACKUP_SERVER_IP:/opt/$ZIMBRA_SERVER_NAME > 3.RSYNC.log
    start services
    i have to write this line coz i needed 10 char..lol
    Raj
    i2k2 Networks
    Dedicated & Shared Zimbra Hosting Provider

  4. #14
    raj's Avatar
    raj
    raj is offline Moderator
    Join Date
    Oct 2005
    Location
    USA, Canada and India
    Posts
    777
    Rep Power
    10

    Default

    [root@zimbraXXX ~]# rsync --version
    rsync version 3.0.6 protocol version 30
    Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
    Web site: rsync
    Capabilities:
    64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, no symtimes
    rsync -avHK --timeout=999 -e ssh /opt/zimbra root@$BACKUP_SERVER_IP:/opt/$ZIMBRA_SERVER_NAME > 1.RSYNC.log
    rsync -avHK --delete --timeout=999 -e ssh /opt/zimbra root@$BACKUP_SERVER_IP:/opt/$ZIMBRA_SERVER_NAME > 2.RSYNC.log
    stop services
    rsync -avHK --delete --timeout=999 -e ssh /opt/zimbra root@$BACKUP_SERVER_IP:/opt/$ZIMBRA_SERVER_NAME > 3.RSYNC.log
    start services
    for rsync..
    1) i have found for long runnig processes i save the output to log for less console text traffic, movreover i have had less disconnection problems for ssh command coz of this..without logging the ssh connection use to timeout or disconnect un-expectedly.
    2) you can rsync 2 or 3 or 4 number of times before stopping services..with hit and trial you will find the best number for min downtime
    3) you can also look for --exclude-from="/opt/zimbrabackup/zimbraskipfolders.txt" and put list of busy/locked folders/files list in it and include that in rsync commands before stopping..that way you can skip big locked files or temp folders..i.e amavis and stuff

    Raj
    Last edited by raj; 03-28-2010 at 06:23 AM.
    i2k2 Networks
    Dedicated & Shared Zimbra Hosting Provider

  5. #15
    ewilen's Avatar
    ewilen is offline Moderator
    Join Date
    Jun 2008
    Location
    Berkeley, CA
    Posts
    1,474
    Rep Power
    9

    Default

    (Not sure why I missed your update, but I just saw it now.)

    Thanks for checking that and for those additional tips. I just installed rsync 3.0.7 on both ends and I'll run some tests.

    Why do you use the -K switch?

  6. #16
    raj's Avatar
    raj
    raj is offline Moderator
    Join Date
    Oct 2005
    Location
    USA, Canada and India
    Posts
    777
    Rep Power
    10

    Default

    -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)
    -v, --verbose increase verbosity
    -H, --hard-links preserve hard links
    -K, --keep-dirlinks treat symlinked dir on receiver as dir

    worked for me for years..first time looked at the man page for definations..

    Raj
    i2k2 Networks
    Dedicated & Shared Zimbra Hosting Provider

  7. #17
    dave_kempe is offline Partner (VAR/HSP)
    Join Date
    Sep 2005
    Location
    Sydney, Australia
    Posts
    291
    Rep Power
    9

    Default

    Quote Originally Posted by ewilen View Post
    If I can piggyback on this thread--even though I use NE--my experience is that rsync can take a while even if you've already copied most of the data. Why? Because there's so much to compare before the copying occurs. Or am I doing something wrong?
    long compare times can be because the OS has run our of buffer cache and the filesystem list has fallen out of the cache, meaning it is read from disk again.
    http://www.solutionsfirst.com.au/hosting/zimbra/
    Australia's premier Zimbra Hosting Partner
    Resellers wanted!

  8. #18
    ewilen's Avatar
    ewilen is offline Moderator
    Join Date
    Jun 2008
    Location
    Berkeley, CA
    Posts
    1,474
    Rep Power
    9

    Default

    Quote Originally Posted by dave_kempe View Post
    long compare times can be because the OS has run our of buffer cache and the filesystem list has fallen out of the cache, meaning it is read from disk again.
    Huh, for some reason I missed your reply. Thanks. This is exactly what I'm beginning to suspect.

    rsync version 3.0.7 doesn't seem to have helped, in fact it may even be slower. What I'm looking into is changing the value of /proc/sys/vm/vfs_cache_pressure as mentioned at Old Nabble - Samba - rsync - sync performance falls off a cliff, but I haven't tried that yet. If that helps, then adding RAM to the system might also speed things up. (I currently have 8 Gb on the primary.)

    raj, how much RAM on your machine? Have you tweaked the buffer/swap settings?

  9. #19
    yogg1 is offline Loyal Member
    Join Date
    Dec 2009
    Posts
    75
    Rep Power
    5

    Default

    Hi

    We use LVM Snapshot for this. Stop the service, make an snapshot and start the service.

    Create an LVM snapshot takes ~ 1sec

    I think our servers are down for ~1 minute. The zimbra daemons take very long to start ^^

    yogg
    Release 7.1.2_GA_3268.UBUNTU8_64 UBUNTU8_64 NETWORK edition.

  10. #20
    lytledd is offline Elite Member
    Join Date
    Dec 2009
    Location
    Michigan
    Posts
    448
    Rep Power
    5

    Default

    Quote Originally Posted by yogg1 View Post
    Hi

    We use LVM Snapshot for this. Stop the service, make an snapshot and start the service.
    And, if you're using an XFS file system, lvcreate will preform an XFS_Freeze and you don't even need to stop the service (I've read).

    Doug
    Ben Franklin quote:

    "Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety."

Page 2 of 2 FirstFirst 12

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. postfix relay=none status=bounced for local mails
    By vdd in forum Administrators
    Replies: 1
    Last Post: 08-06-2009, 08:05 AM
  2. Zimbra Open Source Backup Help
    By ayush1440 in forum Administrators
    Replies: 8
    Last Post: 11-13-2008, 04:56 AM
  3. /tmp filling
    By Nutz in forum Administrators
    Replies: 8
    Last Post: 02-22-2008, 02:00 AM
  4. Replies: 8
    Last Post: 02-27-2007, 04:10 AM
  5. Unable to start tomcat
    By chanck in forum Administrators
    Replies: 11
    Last Post: 06-11-2006, 12:58 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •