Simple backup script for OSE version

[uxbod]

So, outside of rsync were you able to SSH across to the new server without specifying either a password or passphrase ? Just wondering whether it is how rsync handles SSH.

[cornbread]

I haven't had time to try the script but ssh and rsync over ssh I have working fine on a few systems without ssh passkey's.

[jpbuse]

What is the difference between the full_backup and diff_backup portion of this script other than the hotsync? It is still backing up the complete /opt/zimbra directory regardless, correct?

[ArcaneMagus]

Correct it still fully backs up the entire zimbra directory. The difference is that the "diff" backup runs the rsync while zimbra is still running and then shuts down zimbra and runs the rsync again. This will cause pretty much nothing but the log files to have been changed between the two syncs (and any email's received since the live rsync occurred)

If you have LVM setup on your zimbra partition then you have a much better option available. With LVM you can simply shutdown zimbra, take a snapshot, and start zimbra again. Run your backup on the snapshot, and then remove it when done. Your total downtime is only the time that it takes your server to shutdown and then start zimbra (the snapshot is virtually instant).

https://support.ginsys.be/svn/zimbrackup/ This is an excellent script if you do have LVM setup on your system.

[jpbuse]

Thanks for the great and simple script. I made a few edits to allow for an archive directory and rsync to a remote server. I only made the edits to the diff_backup and msgdiff_backup functions. I also added a timestamp variable before the calls so the filename would remain consistent throughout the backup, archive and sync.

#!/bin/bash

# This script will do various backups of Zimbra depending on wich you choose,
# most of them are cold backups except the msg backup wich hot copies the 'store'
# folder for possible individual mail retrival. Be sure to change the variables below
# to point to where you got the zimbra folder and where you want it to be backed up to.
#
# This script must be run as root or a user with equal privileges or it will not work.
#
# Usage: ZimBackup.sh full	- For full backup (Cold)
#	 ZimBackup.sh diff	- For diffrential backup (Cold)
#	 ZimBackup.sh msgfull	- For complete message backup (Hot)
#	 ZimBackup.sh msgdiff	- For diffrential message backup (Hot)
#
# When you run this script via crontab be sure to add '> /dev/null 2>&1' at the end
# of the script like below or the tar command will fail for no apparent reason.
# 00 12 * * * ZimBackup.sh full > /dev/null 2>&1
#
# Bits and pieces was adopted from a script created by Daniel W. Martin, 9 Sept 2007
# Free to use and free of any warranty!  Marcus Uddenhed, 3 feb 2008
#
# Last updated: 2008-04-01

ZimInstPath=/opt
ZimHome=zimbra
ZimBackupPath=/home/backups
ZimTempPath=/home/backups/tmp
ZimArchive=/home/backups/archive

rsyncUser=myuser
rsyncHost=myhost
rsyncPath="~/sync/"
rsyncBWLimit=400

# Do not change anything below this line unless you know what you are doing

pre_check() {
	# Check to see if the tmp folder exists, and create if not
	mkdir -p $ZimTempPath
	# Check to see if the archive directory exists, and create if not
	mkdir -p $ZimArchive
	mkdir -p $ZimArchive/full
	mkdir -p $ZimArchive/msg
	
	# Check which zimbra version that is installed, for recovery purpose 
	sudo -u zimbra $ZimInstPath/$ZimHome/bin/zmcontrol -v > $ZimTempPath/zimbra_version.txt
}

full_backup() {
	# Removing possible old zimbra backup folder
	rm -r -f $ZimTempPath/$ZimHome
	
	# Stopping Zimbra
	/etc/init.d/zimbra stop
	sleep 20
	
	# Backing up zimbra folder
	cp -rv $ZimInstPath/$ZimHome $ZimTempPath/
	
	# Starting Zimbra
	/etc/init.d/zimbra start
	
	# Compressing backup for space reduction
	tar -zcvf $ZimBackupPath/ZimBackupTypeFull_`date +%Y%m%d%H%M`.tar.gz -C $ZimTempPath $ZimHome zimbra_version.txt
}

diff_backup() {
	# Hot sync before shutdown on zimbra folder
	rsync -avHK --delete $ZimInstPath/$ZimHome $ZimTempPath/
	
	# Stopping Zimbra
	/etc/init.d/zimbra stop
	sleep 20
	
	# Cold sync of zimbra folder
	rsync -avHK --delete $ZimInstPath/$ZimHome $ZimTempPath/
	
	# Starting Zimbra
	/etc/init.d/zimbra start
	
	# Get timestamp for file naming
	timestamp=`date +%Y%m%d%H%M`

	# Compressing backup for space reduction and removing unpacked folder
	tar -zcvf $ZimBackupPath/ZimBackupTypeDiff_$timestamp.tar.gz -C $ZimTempPath $ZimHome zimbra_version.txt

	# Move to archive directory
	mv $ZimBackupPath/ZimBackupTypeDiff_$timestamp.tar.gz $ZimArchive/full
	
	# Sending offsite via rsync w/ bandwidth limitation
	rsync -avHK -e ssh --bwlimit=$rsyncBWLimit $ZimArchive/full/ZimBackupTypeDiff_$timestamp.tar.gz $rsyncUser@$rsyncHost:$rsyncPath
}

msgfull_backup() {
	# Removing possible old store backup folder
	rm -r -f $ZimTempPath/store
	
	# Make dir for hot sync
	mkdir -p $ZimTempPath/store
	
	# Hot sync of mailbox messages
	rsync -avHK --delete $ZimInstPath/$ZimHome/store/0 $ZimTempPath/store/
	
	# Compressing store folder for space reduction
	tar -zcvf $ZimBackupPath/ZimBackupTypeMsg_`date +%Y%m%d%H%M`.tar.gz -C $ZimTempPath store zimbra_version.txt
}

msgdiff_backup() {
	# Make dir for hot sync
	mkdir -p $ZimTempPath/store
	
	# Hot sync of mailbox messages
	rsync -avHK --delete $ZimInstPath/$ZimHome/store/0 $ZimTempPath/store/
	
	# Get timestamp for file naming
	timestamp=`date +%Y%m%d%H%M`

	# Compressing store folder for space reduction
	cd $ZimTempPath
	tar -zcvf $ZimBackupPath/ZimBackupTypeMsg_$timestamp.tar.gz -C $ZimTempPath store zimbra_version.txt
	
	# Move to archive
	mv $ZimBackupPath/ZimBackupTypeMsg_$timestamp.tar.gz $ZimArchive/msg
	

}

case $1 in
full)
pre_check
full_backup
;;
diff)
pre_check
diff_backup
;;
msgfull)
pre_check
msgfull_backup
;;
msgdiff)
pre_check
msgdiff_backup
;;
*)
echo "Usage: ZimColdBackup.sh {full|diff|msgfull|msgdiff}"
;;
esac

[soulskater]

Nice to see that the script is being used unmodified/modified doesn't matter. ;-)

jpbuse / I will look at your script and see if your changes might even fit me and if so i will add it to the script with a cred line with your name. ;-)

ArcaneMagus is correct, the full backup cleans out the tmp folder under your backup folder and copies every file/folder in your zimbra install while diff backup syncs the installation folder once before shutdown and once after, the diff is a bit quicker when there are many files that has been changed since it minimize the downtime for the system.

[jpbuse]

Its a nice and simple script, often the best type! The way I am currently handling the backup (as a test) is for my backup server to rsync the zimbra data to local storage. This basically using the diff function but called from a remote server. We're still using Retrospect (gasp) which simply backups the zimbra sync for us. Below is the basics of the script. This is called from my backup server, not on the Zimbra server. My central backup server goes out to many different servers each day and does a sync to a local storage device. Then Retrospect simply backups the local storage device.

#!/bin/bash
RSYNC=/usr/bin/rsync
RSYNC_ARGS="-avHPK --exclude=whatever"
archiveDir=/Volumes/raid3-2/server_backups

###### ZCS server ######
remoteServer=zcs.mydomain.com
remotePath=/opt
localDir=$archiveDir/zmail
user=root

# Make local backup directory
mkdir -p $localDir


# Hot sync before shutdown on zimbra folder
$RSYNC $RSYNC_ARGS --delete -e ssh "$user@$remoteServer:$remotePath" $localDir

# Shutdown Zimbra
ssh $user@$remoteServer '/etc/init.d/zimbra stop'
sleep 20

# Cold sync of zimbra folder
$RSYNC $RSYNC_ARGS --delete -e ssh "$user@$remoteServer:$remotePath" $localDir

# Start Zimbra
ssh $user@$remoteServer '/etc/init.d/zimbra start'

[soulskater]

Looks nice and simple, kinda like it complex and... nah just kidding, looking good.

/soulskater

[soulskater]

Script has been updated, se below for information and link to script.

Changes as of 2008-05-18:
Now it is possible to decide how many local copies of backup files to keep based on file timestamps. Lets say that we do backup once a night at 12 and want to keep to 2 successive copies locally and delete the rest, the by setting ZimDeleteTimeSet to 3000, which is 2880min which is equal to 2 days + 120min as a margin in case the backup should take a little longer than usual, we will keep the number of files we want locally. The time is set in minutes to ensure some degree of flexibility.

Also added some help in the script for user configurable settings.

Download here: Script Download

[soulskater]

Yet another script upgrade.

Changes as of 2008-06-10:
Help inside the script for configuration options.
Logging capability, now you can log your backup either with minimal information or in a verbose mode that throws pretty much anything into the log useful for when trying to find backup problems.

Download: Here

If you find my script useful don't hesitate to post in my blog stating so, it would make me a happy camper.