[SOLVED] Yet Another Backup Script for Community Version

[ramjimh]

Hi

Added a Redhat compatible version on the download site, for now. Till the next release is out where all distros should be compatible (I hope).

Cheers
heinzg

P.S. thank you for your feed back!

where can i get this script.If available kindly send me to ramjimh@gmail.com

with regards,

ramji

[gvillaran]

Nice script, working really well, i only need to know a former restore method now, if anyone have the instructions on how to do it i will appreciate the information.

Again thanks for the effort on this, is really a good work guys

[Koffie]

PLEASE HELP!!

I'm having SERIOUS trouble trying to get the zmDBbac.sh script to work, please have a look at the error

Had to kill some left over procs...
Doing a fast cold sync...

Creating db_dumps directory...
Backing up Zimbra DBs...
-su: SYNC_DIR: command not found
There was an error running DB backup script! Aborting DB backup. Continuing main backup
Reinstating Zimbra's crontab...
Starting Zimbra...
Host xxx.xxx.xxx
	Starting ldap...
Required variable not set!
Required variable: SYNC_DIR OR makeraw
Usage:
Backup:	       /path/to/zmDBbac.sh /path/to/sync/dir OR
Raw converson: /path/to/zmDBbac.sh makeraw db.sql db-raw.sql

Done.

Please tell me that I am just being a n00b and not setting a variable somewhere, I have had a look but can't seem to find where it needs to be set.

Thanks in advance!

EDIT:!!!!

NEVER MIND, I made a syntax error, & instead of $

It is all working now

Thanks for the script LaFong!

[LaFong]

Nice script, working really well, i only need to know a former restore method now, if anyone have the instructions on how to do it i will appreciate the information.

General restore:
[SOLVED] Yet Another Backup Script for Community Version

Open Source Edition Backup Procedure - Zimbra :: Wiki

These instructions do not cover how to extract dar files, if that's what you're using. For those, you restore the full first, then all the diffs to the same location, in order from earliest to last. You usually would not restore to your current Zimbra path, but to a temporary location. Then use the general restoring instructions from the wiki. I may write a little script to automate restoring dars.

Not part of the main script is more granular backup/restore:
User restore:
[SOLVED] Yet Another Backup Script for Community Version

Single folder restore:
[SOLVED] Yet Another Backup Script for Community Version

[gvillaran]

Thanks LaFong, yeah im using the DAR method but i never worked with dar files before.

so i looked in google for how to work with dar files and found it, but im getting errors while i
try to list the content, the program ask me for a file that i cant find anywhere, i will read
more about dar files today.

Would be really great to have a script to restore from dar method.

Thanks for your help and nice work

[LaFong]

Thanks LaFong, yeah im using the DAR method but i never worked with dar files before.

so i looked in google for how to work with dar files and found it, but im getting errors while i try to list the content, the program ask me for a file

Typically, it's because you don't use the full file name, just the "basename". So if a file is named "15_ZimbraBackup_20110411_FULL.1.dar" and you want to see its contents, leave out the last two fields (.1.dar).

[root@mail /]# dar -l 15_ZimbraBackup_20110411_FULL

Would be really great to have a script to restore from dar method.

Here you go. The usual disclaimers apply: I have not tested thoroughly, it is offered as-is, no warranties are implied or apply.

#!/bin/bash
# Script to extract a set of Zimbra FULL and DIFF dar files to a destination directory.
# Run the script with no parameters to get usage information.

DAR_BIN=`whereis dar | awk '{print $2}'`
if [ -n "$2" ] ; then
   DAR_DIR=`echo $1 | sed 's/\/$//'`
   DEST_DIR="$2"
   FULL_FILE=`ls "$DAR_DIR"/* | grep FULL | grep dar$`
   if [ -n "$FULL_FILE" ] ; then
      if [ ! -d "$DEST_DIR" ] ; then
        echo
        printf "$DEST_DIR does not exist. Would you like it to be created? (y or n): "
        read answer
        case $answer in
           y|Y)
               mkdir "$DEST_DIR"
               if [ "$?" -ne "0" ] ; then
                  echo "Error creating destination directory, aborting. Check your destination path."
                  exit 1
               fi
             ;;
             *)
               echo "No directory available to extract files to, aborting script."
               exit 1
             ;;
        esac
      fi
      FULL=`echo ${FULL_FILE%.*.*}`
      echo "FULL is $FULL"
      $DAR_BIN -x "$FULL" -w -R "$DEST_DIR"
   else
      echo "No FULL file in the DAR directory, aborting!"
      exit 1
   fi
   DIFF_FILE=`ls -tr "$DAR_DIR"/* | grep DIFF | grep dar$ | tail -1`
   if [ -n "$DIFF_FILE" ] ; then
      DIFF=`echo ${DIFF_FILE%.*.*}`
      echo "Diff file is $DIFF"
      $DAR_BIN -x "$DIFF" -wa -R "$DEST_DIR"
   fi
   echo "All done."
else
   printf "\n\n"
   echo 'Not enough parameters.

Usage: SCRIPT_PATH DAR_DIR DEST_DIR

Where SCRIPT_PATH is the path to this script, DAR_DIR is the full path to your dar files,
and DEST_DIR is the full path to the place where you want to put the restored data.
For example:

/usr/local/sbin/zmDARrestore.sh /Backup/zimbra_dars /mnt/restore/zimbra

THINGS TO NOTE:
     * The parent path in DEST_DIR must already exist. For example, using a DEST_DIR of /mnt/restore/zimbra, /mnt/restore must already exist.
      (The restore could be many GB, so we want to be sure that the path (mount point) has enough room.)
     * Be SURE you have enough room in DEST_DIR for your restored files. Avoid restoring to the / partition.
     * The script will create the DEST_DIR for you if it does not exist, and you have write permission.
     * Be careful where you restore Zimbra, the DEST_DIR. You do NOT want to restore to /opt/zimbra if that directory already exists.
     * It assumes you have only one FULL, and one DIFF for that FULL, in the DAR_DIR. Move files as necessary to meet this precondition.
       Move the DIFF you want to use, in order to restore to a specific date. Usually, though, you would use the latest DIFF. If there is
       more than one DIFF in the DAR_DIR, it will use the latest one.
     * It assumes you use the default $FULL_PREFIX ("FULL") and $DIFF_PREFIX ("DIFF").'
printf "\n\n\n"
fi

Here are the THINGS TO NOTE from the script usage section:

• The parent path in DEST_DIR must already exist. For example, using a DEST_DIR of /mnt/restore/zimbra, /mnt/restore must already exist.
  (The restore could be many GB, so we want to be sure that the path (mount point) has enough room.)
• Be SURE you have enough room in DEST_DIR for your restored files. Avoid restoring to the / partition.
• The script will create the DEST_DIR for you if it does not exist, and you have write permission.
• Be careful where you restore Zimbra, the DEST_DIR. You do NOT want to restore to /opt/zimbra if that directory already exists.
• It assumes you have only one FULL, and one DIFF for that FULL, in the DAR_DIR. Move files as necessary to meet this precondition. Move only the DIFF you
  want to use, in order to restore to a specific date. Usually, though, you would use the latest DIFF. If there is more than one DIFF in the DAR_DIR, it will use the
  latest one.
• It assumes you use the default $FULL_PREFIX ("FULL") and $DIFF_PREFIX ("DIFF").'

[gvillaran]

Thats great Lafong....i tested the script today, the full backup was restored fine with no warnings.....but when restoring de diff's the program started to ask me this :

/tmp/restore/store/0/11/msg/0/1040-1703.msg must be removed, but does not match expected type, remove it anyway ? [return = OK | Esc = cancel]
Continuing...

that was like 20 times.....i just hit return for testing purposes and the script ended.

i guess something is wrong with the diff process?

Thank you very much for your effort.

[Lallo]

Hello,
i'm trying to understand what is the bottleneck of this backup procedure.

In the log I have output like these:
...
"Service down time was - Hr:0 Min:6 Sec:3"
...
full Zimbra Backup ended at: 17:29
Backup took Hr:15 Min:59 Sec:54 to complete

Since I'm using DRCP to copy the DAR on another server I was thinking that the ssh transfer time was the main reason for the long backup (I was blaming poor networking). I'm not so sure anymore therefore I've put some echo "now is `date`" in the script to better investigate.

In your experience how long does the full backup procedure take and what is the bottleneck? I obviously can't imagine what would happen if another instance of this script is started before the other has ended.

[LaFong]

Thats great Lafong....i tested the script today, the full backup was restored fine with no warnings.....but when restoring de diff's the program started to ask me this :

/tmp/restore/store/0/11/msg/0/1040-1703.msg must be removed, but does not match expected type, remove it anyway ? [return = OK | Esc = cancel]
Continuing...

that was like 20 times.....i just hit return for testing purposes and the script ended.

i guess something is wrong with the diff process?

Ugh. Yes. Sorry. I forgot that the main script does a complete diff every time, not an incremental one. So, every diff is based off the full and, if you have at least one diff, you only need the full dar and one diff in order to restore. That diff will usually be latest diff for that particular full. I changed the script to reflect that. These facts also make manual restore much easier:
For a full named 15_ZimbraBackup_20110411_FULL.1.dar and latest diff named 15_ZimbraBackup_20110414_DIFF.1.dar and

dar -x /Backup/Zimbra_dars/15_ZimbraBackup_20110411_FULL -w -R /mnt/restore
dar -x /Backup/Zimbra_dars/15_ZimbraBackup_20110414_DIFF -w -R /mnt/restore

You must always use a diff which is based off the full.

[gvillaran]

oh aright, can u post the script again? with the corrections, so i can go and test it again?

Thanks