Zimbra OSE backup without stopping the services

[sgnaus]

Hi all,

we have a problem: we must set a backup procedure for our zimbra open source edition, single server installation.

All of the procedures i've seen needs to stop the services for the entire backup operation; this is unacceptable for us because our /opt/zimbra directory is 30gb big, and a simple local rsync process takes 1:30 hours to complete.

We cannot take the server down for that period, so i ask you if there's an alternative procedure to backup the server without stopping services or stopping them for a little time period.

Thank you.

[uxbod]

LVM and splitting a mirror ?

[sgnaus]

but with lvm i must stop the services anyway isnt'it? that's my problem, i cannot stop services...

[uxbod]

stopping them for a little time period

stop the services, split the mirror, and start them again.

[sgnaus]

sorry for this, but please could you specify in a better way what you're meaning for "split the mirror".

thanks a lot.

[liverpoolfcfan]

If you look at the script in this thread

[SOLVED] Yet Another Backup Script for Community Version

it does stop the services - but - for a very short period of time.

It does an rsync while the services are running - to get most of the backup data in place - then stops the services and does a second rsync to backup the remaining files that changed since the first rsync was begun.

At that point it restarts the mail services before actually performing the rest of the backup processes - in this case using dar and remote copy.

In my case - with approx 1.8GB of mailbox data the mail is offline for 2 mins and 10 seconds. I'm not sure how much the size of mailbox data affects the startup times.

[raj]

You only need 2-3 min of service stop/start time no matter if you have Tera bytes of data..
key is to rsync multiple times to shorten the "delta" sync when you stop services.
i.e:
rsync
rysnc
rysnc
..
stop services
rsync
start services

or go vmware route and if you have correct vmware lisence you can take HOT backup of full VM any times you like, VCB/Stoarge API can do incremental backups too..
if you have large number of users and OSS version fuctions are enough for you then vmware + OSS will cost less than just NE with Backup for same amount of users..much less

Raj
Raj

[sgnaus]

Originally Posted by liverpoolfcfan

If you look at the script in this thread

[SOLVED] Yet Another Backup Script for Community Version

it does stop the services - but - for a very short period of time.

It does an rsync while the services are running - to get most of the backup data in place - then stops the services and does a second rsync to backup the remaining files that changed since the first rsync was begun.

At that point it restarts the mail services before actually performing the rest of the backup processes - in this case using dar and remote copy.

In my case - with approx 1.8GB of mailbox data the mail is offline for 2 mins and 10 seconds. I'm not sure how much the size of mailbox data affects the startup times.

thank you a lot man, i'll check it out

[ewilen]

Originally Posted by raj

You only need 2-3 min of service stop/start time no matter if you have Tera bytes of data..
key is to rsync multiple times to shorten the "delta" sync when you stop services.
i.e:
rsync
rysnc
rysnc
..
stop services
rsync
start services

If I can piggyback on this thread--even though I use NE--my experience is that rsync can take a while even if you've already copied most of the data. Why? Because there's so much to compare before the copying occurs. Or am I doing something wrong?

[raj]

sample backup log for our custom backup script for ~100GB actual data in this server..only have zimbra + 600MB plain centos install.
[root@zimbraXXX ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 202G 93G 99G 49% /
/dev/sda1 99M 17M 78M 18% /boot
tmpfs 1014M 0 1014M 0% /dev/shm

Thu Mar 25 02:46:57 EDT 2010 : Backup Process Started
Thu Mar 25 02:46:57 EDT 2010 : Zimbra Mysql Databases Backup Dump Started
Thu Mar 25 02:49:51 EDT 2010 : Zimbra Mysql Databases Backup Dump Finished
Thu Mar 25 02:49:51 EDT 2010 : Zimbra Mysql Databases Backup Dump Time Taken: 0 hours 2 minutes 54 seconds
Thu Mar 25 02:49:51 EDT 2010 : Zimbra LDAP Databases Backup Dump Started
Thu Mar 25 02:49:56 EDT 2010 : Zimbra LDAP Databases Backup Dump Finished
Thu Mar 25 02:49:56 EDT 2010 : Zimbra LDAP Databases Backup Dump Time Taken: 0 hours 0 minutes 5 seconds
Thu Mar 25 02:49:56 EDT 2010 : Initial File Sync Started
Thu Mar 25 02:56:16 EDT 2010 : Initial File Sync Last 2 Log Lines:
sent 938450506 bytes received 4275600 bytes 2477598.18 bytes/sec
total size is 147472277364 speedup is 156.43
Thu Mar 25 02:56:16 EDT 2010 : Initial File Sync Finished
Thu Mar 25 02:56:16 EDT 2010 : Initial File Sync Time Taken: 0 hours 6 minutes 20 seconds
Thu Mar 25 02:56:16 EDT 2010 : Second File Sync Started
Thu Mar 25 02:59:23 EDT 2010 : Second File Sync Last 2 Log Lines:
sent 50608948 bytes received 1235691 bytes 276504.74 bytes/sec
total size is 147476019978 speedup is 2844.58
Thu Mar 25 02:59:23 EDT 2010 : Second File Sync Finished
Thu Mar 25 02:59:23 EDT 2010 : Second File Sync Time Taken: 0 hours 3 minutes 7 seconds
Thu Mar 25 02:59:23 EDT 2010 : Stopping Zimbra Services
Host zimbra3.ocpwebserver.com
Stopping stats...Done
Stopping mta...Done
Stopping spell...Done
Stopping snmp...Done
Stopping archiving...Done
Stopping antivirus...Done
Stopping antispam...Done
Stopping imapproxy...Done
Stopping mailbox...Done
Stopping logger...Done
Stopping ldap...Done
Thu Mar 25 03:00:34 EDT 2010 : Incremental File Sync Started
Thu Mar 25 03:03:42 EDT 2010 : Incremental File Sync Last 2 Log Lines:
sent 150317644 bytes received 1634886 bytes 806114.22 bytes/sec
total size is 150554389283 speedup is 990.80
Thu Mar 25 03:03:42 EDT 2010 : Incremental File Sync Ended
Thu Mar 25 03:03:42 EDT 2010 : Services Stopped to do Incremental File Sync for : 0 hours 4 minutes 19 seconds
Thu Mar 25 03:03:42 EDT 2010 : Starting Zimbra Services
Host zimbra3.ocpwebserver.com
Starting ldap...Done.
Starting logger...Done.
Starting mailbox...Done.
Starting antispam...Done.
Starting antivirus...Done.
Starting snmp...Done.
Starting spell...Done.
Starting mta...Done.
Starting stats...Done.
Thu Mar 25 03:05:02 EDT 2010 : Checking Zimbra Services
Host zimbra3.ocpwebserver.com
antispam Running
antivirus Running
ldap Running
logger Running
mailbox Running
mta Running
snmp Running
spell Running
stats Running
Thu Mar 25 03:05:10 EDT 2010 : Backup Process Ended
Thu Mar 25 03:05:10 EDT 2010 : Total Time Taken: 0 hours 18 minutes 13 seconds
Thu Mar 25 03:05:10 EDT 2010 : -----------------------------------------------------

Raj