[SOLVED] Yet Another Backup Script for Community Version

[taenia]

The answer is: Yes. Using CRON, I don't receive empty mails.

But there's always a problem:

At the end of the mail I've:

:::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::
Full Zimbra Backup ended at: 11:36
Backup took Hr:0 Min:11 Sec:15 to complete
:::::::::::::::::Cheers Osoffice for the script:::::::::::::::::::::::

But when I look at the log, I see:

:::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::
Full Zimbra Backup ended at: 11:36
Backup took Hr:0 Min:11 Sec:15 to complete
:::::::::::::::::Cheers Osoffice for the script:::::::::::::::::::::::
/root/zmbac.sh: line 901: /backup/zimbra/dars/Backuplist-22-August-2008.txt.gz: Permission denied

And the files
33_Zimbra_Backup_22-August-2008_FULL.1.dar
33_Zimbra_Backup_22-August-2008_FULL.1.dar.md5

are sent throught SCP, but not the file :
Backuplist-22-August-2008.txt.gz

[taenia]

So I've run the script tonight in cron, and the only file there was :

Backuplist-23-August-2008.txt

and in there :

Aborting program. User refused to continue while asking: /backup/zimbra/dars/33_Zimbra_Backup_23-August-2008_DIFF.1.dar is required for further operation, please provide the file.

I don't know why, cause I'd run the script in cron before,and I've never get this error...
Could it be cause I ran ./zmbac.sh -f
and tonight, cron run ./zmbac.sh -d but as there were no *FULL.1.dar, so it doesn't work ?

Thanks

[bbarrons]

I get an error when I try to install I tried sudo, as root, as zimbra user.... I know it must be something simple but I am lost.....
here is what I get
zimbra@ms2:/home/bbarrons/Documents$ sh ./zmbac.sh -install
[: 118: Illegal number:
touch: cannot touch `/var/log/zim_backup.log': Permission denied
./zmbac.sh: 124: Syntax error: "(" unexpected
this was as zimbra user as root I get
root@ms2:/home/bbarrons/Documents# ./zmbac.sh -INSTALL
bash: ./zmbac.sh: Permission denied
root@ms2:/home/bbarrons/Documents# sh ./zmbac.sh -INSTALL
[: 118: Illegal number:
./zmbac.sh: 124: Syntax error: "(" unexpected
What am I doing wrong?
thanks
Bill

[chrishewitt]

Have you tried looking at line 124 of the script for the unexpected open bracket "(" that the script is barfing on?

[bbarrons]

yep, I removed brackets at that point and then it went to the next line with brackets and so on.... ea line that had brackets it barfed. I downloaded it again and then also cut and pasted it from another location..... I figured I must be executing it wrong.....

[Thraxis]

For those of you getting the error on line 901 and/or Full Backup e-mails with no file list attached. It seems there are a couple issues with the script.

#1. It the $UUENCODE variable is only given a value during the installation section of the script.

around line 527 after:

    MD5SUM_BIN=`whereis md5sum | awk '{print $2}'`
    if [ ! -e "$MD5SUM_BIN" ]
    then
         echo "Please install "md5sum"!"
         echo "Try running the script with --INSTALL"
         cat $LOG | mail -c $EMAILCC -s "Zimbra backup error on `hostname --fqdn`" $EMAIL
         exit 1
    fi

Add the following:

    UUENCODE_BIN=`whereis uuencode| awk '{print $2}'`
    if [ ! -e "$UUENCODE_BIN" ]
    then
         echo "Please install "uuencode"!"
         echo "Try running the script with --INSTALL"
         cat $LOG | mail -c $EMAILCC -s "Zimbra backup error on `hostname --fqdn`" $EMAIL
         exit 1
    fi

Then go down to around line 908 (used to be 901) and change:

(cat $LOG; $UUENCODE "$ARCHIVEDIR"Backuplist-"$BACKUPDATE".txt.gz "$ARCHIVEDIR"Backuplist-"$BACKUPDATE".txt.gz) | mail -c $EMAILCC \

to

(cat $LOG; $UUENCODE_BIN "$ARCHIVEDIR"Backuplist-"$BACKUPDATE".txt.gz "$ARCHIVEDIR"Backuplist-"$BACKUPDATE".txt.gz) | mail -c $EMAILCC \

[chrishewitt]

I've made some modifications to the original script source:

First: Prefixing the Backuplist filename with the $BACKUPWEEK variable to make manual moves of the non-current-week files to the "old" directory so they're held alongside the .dar and .dar.md5 files (which the script moves automatically). e.g. "mv 36* ../old" is all that's required.

# over view of all the files which where backed up
    echo
    echo "Create file listing from archive"
    if [ $CRYPT = "yes" ]
	then
	    KEY=`cat "$PASSDIR""$PASSFILE"`
	    nice -19 $DAR_BIN -K bf:$KEY -l "$ARCHIVEDIR""$BACKUPWEEK"_"$BACKUPNAME"_"$BACKUPDATE"_"$BACKUPTYPE_D" -Q\
	    > "$ARCHIVEDIR""$BACKUPWEEK"_Backuplist_"$BACKUPDATE".txt && gzip -9 "$ARCHIVEDIR""$BACKUPWEEK"_Backuplist_"$BACKUPDATE".txt
	else        
	    nice -19 $DAR_BIN -l "$ARCHIVEDIR""$BACKUPWEEK"_"$BACKUPNAME"_"$BACKUPDATE"_"$BACKUPTYPE_D" -Q\
	    > "$ARCHIVEDIR""$BACKUPWEEK"_Backuplist_"$BACKUPDATE".txt && gzip -9 "$ARCHIVEDIR""$BACKUPWEEK"_Backuplist_"$BACKUPDATE".txt
    fi

Second: I've modified the timer report section of the script so that it calculates the elapsed time correctly. The original script often reported nonsense elapsed times of 1hr 93mins etc.

    # Script Timer
    STOPTIME=(`date +%s`)
    RUNTIME=$(expr $STOPTIME - $STARTTIME)
    echo
    echo "::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::" 
    echo "Diff Zimbra Backup ended at: `date +%H:%M`" 
    echo "Backup took $(date -d "1970-01-01 $RUNTIME sec" +%H:%M:%S) to complete" 
    echo "::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::"

Both modifications need to be made twice; once each in the FULL and DIFF sections of the script, with minor differences between edits to make them appropriate for the section.

Christian

[chrishewitt]

ok.. more changes:

First; with reference to my last post. If you change the Backuplist filename to be prefixed with the week number AND Zimbra_ the routine that automates file move from current to old directories works on all files, not just the .dar and .md5 files. e.g. 36_Zimbra_Backuplist_13-September-2008.txt.gz.

Second; to enable daily log files to be sent. Find the following line in the FULL section of the script (modified as per recent posting to prevent the permissions error from occurring and the file renaming above):

(cat $LOG; $UUENCODE_BIN "$ARCHIVEDIR""$BACKUPWEEK"_Zimbra_Backuplist_"$BACKUPDATE".txt.gz "$ARCHIVEDIR""$BACKUPWEEK"_Zimbra_Backuplist_"$BACKUPDATE".txt.gz) | mail -c $EMAILCC \
    -s "Zimbra Diff Backup Log on `hostname --fqdn`" $EMAIL

Notice that you found this line immediately below the section that concludes the script by telling you how long the run-time was? .. paste this line below the exact same point in the DIFF section (scroll right to the bottom), e.g:

# Script Timer
    STOPTIME=(`date +%s`)
    RUNTIME=$(expr $STOPTIME - $STARTTIME)
    echo
    echo "::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::" 
    echo "Diff Zimbra Backup ended at: `date +%H:%M`" 
    echo "Backup took $(date -d "1970-01-01 $RUNTIME sec" +%H:%M:%S) to complete" 
    echo "::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::"
    (cat $LOG; $UUENCODE_BIN "$ARCHIVEDIR""$BACKUPWEEK"_Zimbra_Backuplist_"$BACKUPDATE".txt.gz "$ARCHIVEDIR""$BACKUPWEEK"_Zimbra_Backuplist_"$BACKUPDATE".txt.gz) | mail -c $EMAILCC \
    -s "Zimbra Diff Backup Log on `hostname --fqdn`" $EMAIL
    echo

Don't forget to insert the UUENCODE= lines (as per another recent posting) into the DIFF section to prevent the "901" error.

Now.. AS ROOT, modify root's crontab (crontab -e) and edit the second line that schedules DIFF backups and log rotation so that after the -d there is only one > instead of >>:

30 1 * * 1     /bin/bash     /opt/zmbac.sh -f > /var/log/zim_backup.log 2>&1
30 1 * * 2-7   /bin/bash     /opt/zmbac.sh -d > /var/log/zim_backup.log 2>&1

Save the crontab. You should now receive the log file daily, and the log file should contain only that days output. If you omit the crontab change you generate a single log file each week with each daily run appending itself to the bottom of the report. Personally I prefer to have a new logfile each day so there is less to scroll past to see that the script ran okay (albeit you get more emails to read).

Christian

[devil]

#!/usr/bin/perl

use strict;

#Krasi Nachev , GPL
#backup script

my $userhost = 'root@domain.com'; #define user and host
chomp (my $date = `date +%d%b`);
my %hash = (
remdir => '/remote/backup/', #define remote dir
tar => "tar -cvjf",
rsync => "rsync -aP",
ssh => "ssh"
);
my $ref = \%hash;
$ref->{localdir} = '/opt/zimbra'; #define local dir
unless (opendir DH, $ref->{localdir}) {
print "Backup dir $ref->{localdir} not exist \n";
}
else {
print "Backup dir $ref->{localdir} is OK \n";
}
system("/etc/init.d/zimbra stop")

$ref->{baname} = "backup";$ref->{ext} = ".tar.bz2"; $ref->{rm} = "rm -r";



system("$ref->{rsync} $ref->{localdir} $userhost:$ref->{remdir}$ref->{baname}_$date");
sleep 1;
system("$ref->{ssh} $userhost $ref->{tar} $ref->{remdir}$ref->{baname}$date$ref->{ext} $ref->{remdir}$ref->{baname}_$date");
sleep 1;
system("$ref->{ssh} $userhost $ref->{rm} $ref->{remdir}$ref->{baname}_$date");
system("/etc/init.d/zimbra start")

print "Done\n";

[dwmtractor]

Excellent contribution, devil! I like the use of ssh and I'm sure other users will as well. A couple feedback points, taken from when I did my own shell script backup (you may have seen it here):

1. You might want to do a two-step rsync--once before stopping Zimbra services and a second time afterwards. This allows for a much faster sync while Zimbra services are stopped, and consequently has the mail service down for a much shorter period of time.
2. I use

   rsync -avHK --delete

   as my rsync command options (see rsync man page):

-a --archive mode; equals -rlptgoD (no -H,-A,-X)
-v --verbose mode, useful while debugging
-H --preserve hard links
-K --keep-dirlinks, treat symlinked dir on receiver as dir

and --delete to keep the backup file from growing ever-larger as deleted emails otherwise will stay in your backup forever. I didn't realize this till my backup file had grown from 2gb to 8gb with little change in mailstore size. . .you don't wanna go there!

You may wish to consider similar modifications.