[SOLVED] Yet Another Backup Script for Community Version

[dipeshmehta]

No errors found. It works simply through cron. Email received with complete summary.

Anyways, while trying to restore I get following error:

root@mailbox:/opt/zimbra# dar -x /backup/Zimbra_server/42_ZimbraBackup_20101018_FULL
/backup/Zimbra_server/42_ZimbraBackup_20101018_FULL.1.dar is required for further operation, please provide the file. [return = OK | Esc = cancel]

the file is there at the specified path.

Dipesh

[Lallo]

Hello Heinzg,

Thanks for a wonderful script for FOSS Edition.

I just installed ZCS 6.0.8 (64 bit) on Ubuntu 8.04. Installed zmbak_v.0.8.sh and configured its config part to my environment. When I ran it from command line (-f switch), it backs up perfectly, but not logged. zim_backup.log file has timestamp of starting of backup process, but is of 0 size. Similarly, email received is also blank.

Dipesh

Hello,
maybe you are forgetting to redirect output when you execute the script from the command line?

Here is my crontab
30 1 * * 1 /bin/bash /opt/scripts/zmbac.sh -f > /var/log/zim_backup.log 2>&1
30 1 * * 2-7 /bin/bash /opt/scripts/zmbac.sh -d >> /var/log/zim_backup.log 2>&1

As you can see both standard output and standard error are redirected.

[Lallo]

Hello,
I'm using this wonderful script to backup every night about 12 GB of mail (compressed).
I do the full backup on Monday morning and the diff backup until sunday.
My diskspace is obviously getting filled:

[root@mail zimbra_mail_backup]# du -h .
50G ./burn
13G ./zimbra_dars
62G

I also use the scp feature to put the dars on another server (62GB being used also there at the moment).

Is there a way to automatically delete old dars (i.e. to keep just the last two weeks)?

Thanks a lot!

[LaFong]

Hello,
Is there a way to automatically delete old dars (i.e. to keep just the last two weeks)?

I added that to my backup script. See here.

[pebcomputing]

I added that to my backup script. See here.

I do this in crontab on my debian zimbra server:

## Delete email server backups older than 15 days...
55 19 * * 5 root find /backup/zimbrabak/*Zimbra_Backup* -mtime +15 -exec rm {} \;

Of course, this job is exactly what logrotate was made for also.

[LaFong]

I do this in crontab on my debian zimbra server:

## Delete email server backups older than 15 days...
55 19 * * 5 root find /backup/zimbrabak/*Zimbra_Backup* -mtime +15 -exec rm {} \;

Of course, this job is exactly what logrotate was made for also.

Very elegant. Since it deals wth 24 hour periods, it should handle end of year OK too. Logrotate might work too, though the datestamps in the dar names could be problematic. I do like to get a list of successfully deleted archives in my weekly full mail, which is why I went the script route.

[Lallo]

I added that to my backup script. See here.

LaFong your diff is very interesting.

I've looked at it but not implemented it yet in production (scared )
Just a couple of questions:
1) The original script with DRCP="yes" has a feature to copy via ssh the backups on anoter server. Your diff does not take care of removing this part right?
2) Could i still keep my crontab lines unchanged with this? I guess yes, but you know... :-)

[root@mail scripts]# crontab -l
30 1 * * 1     /bin/bash     /opt/scripts/zmbac.sh -f > /var/log/zim_backup.log 2>&1
30 1 * * 2-7   /bin/bash     /opt/scripts/zmbac.sh -d >> /var/log/zim_backup.log 2>&1

Thanks,
Lallo

[LaFong]

LaFong your diff is very interesting.

I've looked at it but not implemented it yet in production (scared )
Just a couple of questions:
1) The original script with DRCP="yes" has a feature to copy via ssh the backups on anoter server. Your diff does not take care of removing this part right?

Do you mean, removing old backups on the remote server? No, it does not. You could change it to do that. You could change the script like this:

+ # Remove previous week of backups from 2 weeks ago
+        TWO_WEEKS_AGO=$(expr "$BACKUPWEEK" \- 2)
+        LASTYEAR=$(expr `date +%Y` \- 1)
+        FINALWEEK=`date -d "$LASTYEAR"1228 +%V`
+        if [ "$BACKUPWEEK" -eq "1" ] ; then
+            TWO_WEEKS_AGO=$(expr "$FINALWEEK" \- 1)
+        elif [ "$BACKUPWEEK" -eq "2" ] ; then
+            TWO_WEEKS_AGO="$FINALWEEK"
+        elif [ "$TWO_WEEKS_AGO" -lt "10" ] ; then
+            TWO_WEEKS_AGO=0"$TWO_WEEKS_AGO"
+        fi
+        TWO_WEEK_BACKUP=`ssh -i /root/.ssh/id_rsa $SSHUSER@$REMOTEHOST "ls -A -1 "$REMOTEDIR" | grep "$TWO_WEEKS_AGO" | grep "$BACKUPNAME" | cut -d _ -f1 | head -n1"`
+        if [ -z "$TWO_WEEK_BACKUP" ]
+        then
+                echo
+                echo "No two-week old backups found"
+                echo
+        else
+            echo
+            echo "Two-week old backup found...old week= "$TWO_WEEKS_AGO" current week= $BACKUPWEEK"
+            echo 
+            for i in `ssh -i /root/.ssh/id_rsa $SSHUSER@$REMOTEHOST "ls -A -1 "$REMOTEDIR""$TWO_WEEKS_AGO"_"$BACKUPNAME"*"`
+                do
+                    ssh -i /root/.ssh/id_rsa $SSHUSER@$REMOTEHOST "rm $i"
+                    if [ "$?" -ne "0" ]
+                    then
+                        echo "Error during delete!"
+                    else
+                        echo "$i deleted"
+                    fi
+                done
+            echo 
+        fi
     fi
+

If you wanted to test first, change the remove ("rm $i") line to just list the file:

                   ssh -i /root/.ssh/id_rsa $SSHUSER@$REMOTEHOST "ls $i"

Or, if you don't need email acknowledgment of deletions, just use the cronjob by pebcomputing on the remote computer, changing the path as needed.

2) Could i still keep my crontab lines unchanged with this? I guess yes, but you know... :-)

Sure.

[Lallo]

Do you mean, removing old backups on the remote server? No, it does not. You could change it to do that. You could change the script like this:

+ # Remove previous week of backups from 2 weeks ago
+        TWO_WEEKS_AGO=$(expr "$BACKUPWEEK" \- 2)
+        LASTYEAR=$(expr `date +%Y` \- 1)
+        FINALWEEK=`date -d "$LASTYEAR"1228 +%V`
+        if [ "$BACKUPWEEK" -eq "1" ] ; then
+            TWO_WEEKS_AGO=$(expr "$FINALWEEK" \- 1)
+        elif [ "$BACKUPWEEK" -eq "2" ] ; then
+            TWO_WEEKS_AGO="$FINALWEEK"
+        elif [ "$TWO_WEEKS_AGO" -lt "10" ] ; then
+            TWO_WEEKS_AGO=0"$TWO_WEEKS_AGO"
+        fi
+        TWO_WEEK_BACKUP=`ssh -i /root/.ssh/id_rsa $SSHUSER@$REMOTEHOST "ls -A -1 "$REMOTEDIR" | grep "$TWO_WEEKS_AGO" | grep "$BACKUPNAME" | cut -d _ -f1 | head -n1"`
+        if [ -z "$TWO_WEEK_BACKUP" ]
+        then
+                echo
+                echo "No two-week old backups found"
+                echo
+        else
+            echo
+            echo "Two-week old backup found...old week= "$TWO_WEEKS_AGO" current week= $BACKUPWEEK"
+            echo 
+            for i in `ssh -i /root/.ssh/id_rsa $SSHUSER@$REMOTEHOST "ls -A -1 "$REMOTEDIR""$TWO_WEEKS_AGO"_"$BACKUPNAME"*"`
+                do
+                    ssh -i /root/.ssh/id_rsa $SSHUSER@$REMOTEHOST "rm $i"
+                    if [ "$?" -ne "0" ]
+                    then
+                        echo "Error during delete!"
+                    else
+                        echo "$i deleted"
+                    fi
+                done
+            echo 
+        fi
     fi
+

Thanks, but with these changes will it still delete also the backups on the original backup server? Or it will delete just the ones on the remote server?

Also could you please post your full script (or via PM) I want to be sure i'm putting the remove backup section in the right branch or "if then".

Thanks a lot!

[Lallo]

I do this in crontab on my debian zimbra server:

## Delete email server backups older than 15 days...
55 19 * * 5 root find /backup/zimbrabak/*Zimbra_Backup* -mtime +15 -exec rm {} \;

Of course, this job is exactly what logrotate was made for also.

Does this work also if find returns a very long list of files? Or could "xargs" become necessary?