[SOLVED] Yet Another Backup Script for Community Version

Printable View

Show 40 post(s) from this thread on one page

01-14-2012, 11:13 PM
LaFong

Quote:

Originally Posted by aarat

The fake backup is happening on SATA Storage (IBM DS3500) Virtualized on Xenserver. However the dars are being created on cifs share mounted on NAS with 100mbps of LAN Connectivity to server.

CIFS isn't the fastest protocol, and 100mbps is meh, but the speed of dar'ing is the probably the biggest speed factor. The mail server is fully functional while dar'ing, so it's not ordinarily a problem.

Quote:

I am also not receving mail for backup as it ends up with /usr/lib/sendmail not found.

I don't know which distro you are using, http://www.zimbra.com/forums/announc...html#post62754, but it seems you do not have sendmail installed. The 'mail' command uses sendmail, usually. Add a softlink pointing to Zimbra sendmail:

Code:

ln -s /opt/zimbra/postfix/sbin/sendmail /usr/lib/sendmail

Or, you can probably just install sendmail, but make sure it is not enabled. You don't want it competing with Zimbra postfix.
Or, if using a Debian/Ubuntu derivative, you might alter /etc/nail.rc: http://www.zimbra.com/forums/adminis...tml#post215050

Quote:

Here's the backup log============================
Sat Jan 14 00:41:52 IST 2012
Performing FULL backup
============================
full Backup started at: Sat Jan 14 00:41:52 IST 2012
Setting date & version Marker into /opt/zimbra/DATE_VERSION.txt
Doing a hotsync of /opt/zimbra/ to /opt/fakebackup/
rsync: stat "/opt/fakebackup/store/0/111/msg/1/.4154-3372.msg.83CGcp" failed: No such file or directory (2)
rsync: rename "/opt/fakebackup/store/0/111/msg/1/.4154-3372.msg.83CGcp" -> "store/0/111/msg/1/4154-3372.msg": No such file or directory (2)
rsync: link "/opt/fakebackup/store/0/118/msg/0/260-14.msg" => store/0/106/msg/0/279-52.msg failed: File exists (17)

These are normal rsync errors, which you are more likely to see with the first full rsync.

Quote:

Creating MD5 Checksum for 02_ZimbraBackup_20120114_FULL.10.dar...
Creating MD5 Checksum for 02_ZimbraBackup_20120114_FULL.11.dar...
Creating MD5 Checksum for 02_ZimbraBackup_20120114_FULL.12.dar...
Creating MD5 Checksum for 02_ZimbraBackup_20120114_FULL.13.dar...
Creating MD5 Checksum for 02_ZimbraBackup_20120114_FULL.14.dar...

I'm not a fan of a whole bunch of dar files for individual backups. This happens because the default size of the ARCHIVESIZE variable by default is only 4GB. This was originally to fit individual dars on DVD, but almost no one does that. I would set it much larger. e.g.

Code:

ARCHIVESIZE="310000M"

If you don't want a single dar to be 300GB, I would still set it much larger than 4GB.
01-15-2012, 12:41 AM
aarat

Mail problem got resolved by adding the line in nail.rc however When doing differnetial backup I am getting this error

Quote:

Writing a diff backup: 02_ZimbraBackup_20120115_DIFF
into: /Backup/zimbra_dars/ with file sizes of max: 4395M
Saving Unencrtyped Archive...
Aborting program. User refused to continue while asking: /Backup/zimbra_dars/02_ZimbraBackup_20120111_FULL
/Backup/zimbra_dars/02_ZimbraBackup_20120114_FULL.1.dar is required for further operation, please provide the file.
Dar had a problem!
01-15-2012, 09:01 AM
kazooless

Failure Each Night

I just started getting this error almost every night. One night in 5 it didn't occur and a DIFF is what was scheduled for each of them. I don't think they type has anything to do with it though based on the error message. I am wondering if I might have some corruption in the mailbox? Here is a clip from the logs. What do you think? (The log failure is the same each night)

============================
Sun Jan 15 01:05:03 PST 2012
Performing DIFF backup
============================
diff Backup started at: Sun Jan 15 01:05:03 PST 2012
Setting date & version Marker into /opt/zimbra/DATE_VERSION.txt
Doing a hotsync of /opt/zimbra/ to /Backup/fakebackup/
Disabling the Zimbra crontab...
Stopping the Zimbra server...

Host mail.domain.com
Stopping stats...Done.
Stopping mta...Done.
Stopping spell...Done.
Stopping snmp...Done.
Stopping cbpolicyd...Done.
Stopping archiving...Done.
Stopping antivirus...Done.
Stopping antispam...Done.
Stopping imapproxy...Done.
Stopping memcached...Done.
Stopping mailbox...Failed.
Stopping mailboxd...done.
Stopping mysqld...failed.

Stopping logger...Done.
Stopping zmconfigd...Done.
Stopping ldap...Done.
zmcontrol shutdown had an error!
01-15-2012, 01:37 PM
LaFong

Quote:

Originally Posted by aarat

Mail problem got resolved by adding the line in nail.rc however When doing differnetial backup I am getting this error

Code:

Writing a diff backup: 02_ZimbraBackup_20120115_DIFF into: /Backup/zimbra_dars/ with file sizes of max: 4395M Saving Unencrtyped Archive... Aborting program. User refused to continue while asking: /Backup/zimbra_dars/02_ZimbraBackup_20120111_FULL /Backup/zimbra_dars/02_ZimbraBackup_20120114_FULL.1.dar is required for further operation, please provide the file. Dar had a problem!

Diffs require the Full, or will fail. It is not finding the Full where it is expecting to find it. Did you move it? What is the path to your Full file? The script is looking for it in /Backup/zimbra_dars.
01-15-2012, 02:15 PM
LaFong

Quote:

Originally Posted by kazooless

I just started getting this error almost every night. One night in 5 it didn't occur and a DIFF is what was scheduled for each of them. I don't think they type has anything to do with it though based on the error message. I am wondering if I might have some corruption in the mailbox? Here is a clip from the logs. What do you think? (The log failure is the same each night)

Stopping mailbox...Failed.
Stopping mailboxd...done.
Stopping mysqld...failed.

Stopping logger...Done.
Stopping zmconfigd...Done.
Stopping ldap...Done.
zmcontrol shutdown had an error!

It's not able to stop mailbox and mysqld. You could check the logs in /opt/zimbra/log. I actually had a similar problem, so I modified the script. I added a function to try service shutdown multiple times. The services always stop for me, eventually.

Code:

@@ -537,6 +541,35 @@ fi } +function do_stop_services { + stopTries=0 + until [ -z `su - zimbra -c $ZM_HOME"bin/zmcontrol status" | grep Running | head -1 | awk '{print $2}'` ] + do + su - zimbra -c $ZM_HOME"bin/zmcontrol stop" + sleep 60 + stopTries=$(expr "$stopTries" + 1) + if [ "$stopTries" -eq "10" ] ; then + echo "Damn thing won't stop" + echo "Restoring crontab" + crontab -u $ZM_USER "$ZM_HOME"crontab.org + echo "Restarting Zimbra services..." + su - "$ZM_USER" -c $ZM_HOME"bin/zmcontrol start" + mail_log + exit 1 + fi + done +} + function do_backup { TYPE=$1

This function is called if the initial shutdown attempt fails:

Code:

#Starting 'service downtime' counter DOWNTIMEA=(`date +%s`) # Stopping Zimbra - echo "Stopping the Zimbra server..." - echo + printf "Stopping the Zimbra server..." su - $ZM_USER -c $ZM_HOME"bin/zmcontrol stop" if [ "$?" -eq "1" ] then echo "zmcontrol shutdown had an error!" - mail_log - exit 1 + echo "Trying to stop services again." + do_stop_services
01-15-2012, 02:16 PM
kazooless

Thanks for the reply. So should I modify my own script? Or should I download the latest?
01-15-2012, 02:24 PM
LaFong

Quote:

Originally Posted by kazooless

Thanks for the reply. So should I modify my own script? Or should I download the latest?

The latest does not have this change. I made several changes to the script, to suit my particular method of use. Either modify the script yourself, or I could make mine available on Dropbox. I added another function for dar rotation, and call my SQL/LDAP dump script, but the other changes are minor.
01-15-2012, 02:26 PM
kazooless

I see. I'm not that great at scripting, but I'll look closely and see if I can figure it out myself. If I can't, I'll post again and take you up on the dropbox offer.

Thanks,

kazooless
01-15-2012, 02:42 PM
kazooless

Okay, I believe I understand how to edit the script with your additions, but before I do, I would like to be sure I understand correctly.

Your first text box is the creation of a new function that is only called if there is a problem. That call is from the edits in the second box, right? The little "+" signs at the beginning of each line are the only lines you are "adding" to the script, right? Likewise, the "-" signs are lines you are deleting.

In the first text box you have some lines before and after the lines with the "+" signs to help find the place in the script where it should be placed.

Am I understanding correctly?
01-15-2012, 05:46 PM
LaFong

Quote:

Originally Posted by kazooless

Your first text box is the creation of a new function that is only called if there is a problem.

Correct.

Quote:

That call is from the edits in the second box, right?

Correct.

Quote:

The little "+" signs at the beginning of each line are the only lines you are "adding" to the script, right? Likewise, the "-" signs are lines you are deleting.

Correct.

Quote:

In the first text box you have some lines before and after the lines with the "+" signs to help find the place in the script where it should be placed.

Correct.

Quote:

Am I understanding correctly?

Yes. Since you've probably set a few variables, make a backup of your original script.

Show 40 post(s) from this thread on one page