Page 2 of 4 FirstFirst 1234 LastLast
Results 11 to 20 of 40

Thread: My Notes for Installing Zimbra 7.1.3 on Ubuntu Server 10.04.3 LTS

  1. #11
    LHammonds's Avatar
    LHammonds is offline Special Member
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    3

    Default Backup - Individual Mailboxes

    Backup - Individual Mailbox

    The 2nd method involves backing up individual mailboxes. This process can occur while the server is online and thus will not cause any downtime. It can be run as often as you want but you should consider the performance impact on the server as well as how long each run will take. I plan to schedule it once per day during non-peak hours.

    /var/scripts/prod/mailbox-backup.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : mailbox-backup.sh
    ## Version       : 1.1
    ## Date          : 2012-01-09
    ## Author        : LHammonds
    ## Purpose       : Backup individual mailbox accounts.
    ## Compatibility : Verified on Ubuntu Server 10.04.3 LTS, Zimbra 7.1.2 - 7.1.3 OSE
    ## Requirements  : Zimbra must be online, must be run as root user.
    ## Run Frequency : Once or more per day.
    ## Exit Codes    : (if multiple errors, value is the addition of codes)
    ##    0 = Success (or non-critical failure)
    ##    1 = Root access failure
    ##    2 = Archive creation failure
    ##    4 = Archive purge error
    ##    8 = configuration error
    ##   16 = Offsite mount failure
    ##   32 = Offsite copy failure
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-10-23 LTH Created script.
    ## 2011-11-05 LTH Move common variables and functions to external file.
    ## 2011-11-06 LTH Additional error checks and better log info.
    ## 2012-01-09 LTH Bugfix f_PurgeOldestArchive
    #############################################
    
    ## Import common variables and functions.
    source /var/scripts/common/standard.conf
    
    ## Define local variables.
    LOGFILE="${TEMPDIR}/mailbox-backup.log"
    HISTORYFILE="${TEMPDIR}/mailbox-backup-size-history.log"
    TARGETDIR="${BACKUPDIR}/mailbox"
    OFFSITEBACKDIR="${OFFSITEDIR}/mailbox"
    ARCHIVEFILE="`date +%Y-%m-%d-%H-%M`_mailbox-all.tar"
    LOCKFILE="${TEMPDIR}/mailbox-backup.lock"
    EXCEPTIONS="spam.ppnvqogp0@${MYDOMAIN};ham.iki6sotcy@${MYDOMAIN};virus-quarantine.qvj6nc_jl@${MYDOMAIN}"
    RETURNVALUE=0
    UCOUNT=0
    ERRORFLAG=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    
    function f_PurgeOldestArchive()
    {
      ## Purpose: Delete the oldest archive on the remote site.
      ## Return values:
      ##    0 = Success
      ##    1 = Cannot delete file
      ##    9 = Configuration error, path empty
    
      ## Variable Error Check. *
      if [ ${OFFSITEBACKDIR} = "" ]; then
        ## Make darn sure the path is not empty since we do NOT
        ## want to start purging files from a random location.
        echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OFFSITEBACKDIR site variable is empty!" >> ${LOGFILE}
        return 9
      fi
      ## Get the name of the oldest file.
      OLDESTFILE=`ls -1t ${OFFSITEBACKDIR} | tail -1`
      FILESIZE=`ls -la ${OFFSITEDIR}/${OLDFILE} | awk '{print $5}' | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'`
      if [ "${OLDESTFILE}" = "" ]; then
        ## Error. Filename variable empty.
        echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OLDESTFILE variable is empty." >> ${LOGFILE}
        return 9
      else   
        FILESIZE=`ls -lak "${OFFSITEBACKDIR}/${OLDESTFILE}" | awk '{ print $5 }' | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'`
        echo "`date +%Y-%m-%d_%H:%M:%S` --- Purging old file: ${OFFSITEBACKDIR}/${OLDESTFILE}, Size = ${FILESIZE} kb" >> ${LOGFILE}
        rm "${OFFSITEBACKDIR}/${OLDESTFILE}"
        if [ -f "${OFFSITEBACKDIR}/${OLDESTFILE}" ]; then
          ## File still exists.  Return error.
          return 1
        else
          return 0
        fi
      fi
    }
    
    function f_cleanup()
    {
      if [ -f "${LOCKFILE}" ]; then
        ## Remove run file since this script is complete.
        rm ${LOCKFILE}
      fi
    }
    
    function f_emergencyexit()
    {
      ## Purpose: Exit script as cleanly as possible.
      ## Parameter #1 = Error Code.
      f_cleanup
      f_sendmail "Zimbra Mailbox Backup Error" "EXIT CODE = ${1}"
      echo "`date +%Y-%m-%d_%H:%M:%S` - Mailbox backup aborted.  EXIT CODE: ${1}" >> ${LOGFILE}
      ## Write script name and error code to the system log.
      logger "${SCRIPTNAME}: ERROR CODE = ${1}"
      exit $1
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    
    ## Requirement Check: Script must run as root user.
    if [ "$(id -u)" != "0" ]; then
      ## FATAL ERROR DETECTED: Document problem and terminate script.
      ERRORFLAG=$((${ERRORFLAG} + 1))
      echo "`date +%Y-%m-%d_%H:%M:%S` - Mailbox backup aborted.  ERROR CODE ${ERRORFLAG}: Script must be run as root user instead of $(whoami)." | tee -a ${LOGFILE}
      f_emergencyexit ${ERRORFLAG}
    fi
    
    if [ -f "${LOCKFILE}" ]; then
      ## Last call to this script is still running or failed to remove its lock file.
      ## As an additional check, see if the file is older than today...if so, we should
      ## probably send an email notification of a problem that may need manual interention.
      FILEDATE=$(stat -c %y ${LOCKFILE})
      FILEDATE=${FILEDATE%% *}
      if [ "${FILEDATE}" != "$(date +%Y-%m-%d)" ]; then
        ## Lock file not created today, might need to be manually deleted.  Send email notification.
        f_sendmail "Zimbra Mailbox Backup Warning" "Warning: This script cannot run if it detects this lock file: ${LOCKFILE}\n\nThis file should only exist while this script is running which should not take more than a day.\n\nSystem Date: $(date +%Y-%m-%d)\nLock File Date: ${FILEDATE}\n\nIf you determine that the file should be removed, do so by typing this command on the server's console:  rm ${LOCKFILE}"
      fi
      exit 0
    else
      ## Create the "script is running" lock file and process the script.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ${HOSTNAME}:${SCRIPTNAME} is currently running." > ${LOCKFILE}
    fi
    
    ## Record the start time of the backup process.
    STARTTIME="$(date +%s)"
    
    echo "`date +%Y-%m-%d_%H:%M:%S` - Individual mailbox backup started." >> ${LOGFILE}
    if [ -d "${TARGETDIR}" ]; then
      ## Purge existing archives.
      rm ${TARGETDIR}/*.tgz 1>/dev/null 2>&1
    else
      ## Make the folder since it does not exist.
      mkdir -p ${TARGETDIR} 1>/dev/null 2>&1
    fi
    for ACCT in `su - zimbra -c "zmprov -l gaa"`
    do
      ## Check to see if current account should be skipped.
      if echo "${EXCEPTIONS}" | grep -q ${ACCT}
      then
        ## Exception found, skip this account.
        echo "" > /dev/null
      else
        ## Backup user account.
        UCOUNT=$((UCOUNT+1))
        ${ZIMBRADIR}/bin/zmmailbox -z -m ${ACCT} getRestURL "//?fmt=tgz" > ${TARGETDIR}/${ACCT}.tgz
        RETURNVALUE=$?
        if [ ${RETURNVALUE} -ne 0 ]; then
          ## Something went wrong.
          echo "`date +%Y-%m-%d_%H:%M:%S` --- Error on ${ACCT}, RETURN VALUE = ${RETURNVALUE}" >> ${LOGFILE}
          ERRORFLAG=$((ERRORFLAG+1))
        else
          ## Calculate archive size.
          MAILBOXSIZE=$(stat -c %s ${TARGETDIR}/${ACCT}.tgz)
          ## Comment out the below line if you do not want details in the log file.
          echo "`date +%Y-%m-%d_%H:%M:%S` --- ${ACCT}, ${MAILBOXSIZE} bytes" >> ${LOGFILE}
        fi
      fi
    done
    echo "`date +%Y-%m-%d_%H:%M:%S` --- ${UCOUNT} accounts processed." >> ${LOGFILE}
    
    ## Calculate mailbox backup time.
    FINISHTIME="$(date +%s)"
    ELAPSEDTIME="$(expr ${FINISHTIME} - ${STARTTIME})"
    HOURS=$((${ELAPSEDTIME} / 3600))
    ELAPSEDTIME=$((${ELAPSEDTIME} - ${HOURS} * 3600))
    MINUTES=$((${ELAPSEDTIME} / 60))
    SECONDS=$((${ELAPSEDTIME} - ${MINUTES} * 60))
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Backup time for ${UCOUNT} mailboxes: ${HOURS} hour(s) ${MINUTES} minute(s) ${SECONDS} second(s)" >> ${LOGFILE}
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Setting file permissions on ${TARGETDIR}/*.tgz" >> ${LOGFILE}
    chmod 0600 ${TARGETDIR}/*.tgz
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Creating a single file for archiving: ${TEMPDIR}/${ARCHIVEFILE}" >> ${LOGFILE}
    tar -cf ${TEMPDIR}/${ARCHIVEFILE} ${TARGETDIR} 1>/dev/null 2>&1
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## Something went wrong.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Error creating ${TEMPDIR}/${ARCHIVEFILE}, Return Value: ${RETURNVALUE}" >> ${LOGFILE}
      ERRORFLAG=$((ERRORFLAG+2))
    fi
    ARCHIVESIZE=$(stat -c %s ${TEMPDIR}/${ARCHIVEFILE})
    if [ ! -f "${HISTORYFILE}" ]; then
      ## Create history log and include header description as 1st line, field separator = space character.
      echo "Date Archive SizeInBytes" > ${HISTORYFILE}
    fi
    ## Append archive size to the history log.
    echo "`date +%Y-%m-%d` ${ARCHIVEFILE} ${ARCHIVESIZE}" >> ${HISTORYFILE}
    
    ## Mount the offsite Windows share folder.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Mounting ${OFFSITEDIR}" >> ${LOGFILE}
    f_mount
    if [ -f ${OFFSITETESTFILE} ]; then
      ## Remote site is online / available.
      if [ ! -d ${OFFSITEBACKDIR} ]; then
        ## Make the backup folder since it does not exist yet.
        mkdir -p ${OFFSITEBACKDIR}
      fi
      if [ -f ${TEMPDIR}/${ARCHIVEFILE} ]; then
        ## Make sure space is available on the remote server to copy the file.
        FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'`
        BACKUPSIZE=`ls -lak "${TEMPDIR}/${ARCHIVEFILE}" | awk '{ print $5 }'`
        if [ ${FREESPACE} -lt ${BACKUPSIZE} ]; then
          ## Not enough free space available.  Purge existing backups until there is room.
          ENOUGHSPACE=0
          while [ ${ENOUGHSPACE} -eq 0 ]
          do
            f_PurgeOldestArchive
            RETURNVALUE=$?
            case ${RETURNVALUE} in
            1)
              ## Cannot purge archives to free up space.  End program gracefully.
              echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Not enough free space on ${OFFSITEBACKDIR} and cannot purge old archives.  Script aborted." >> ${TEMPLOG}
              ## Stop and exit the script with an error code.
              ERRORFLAG=$((${ERRORFLAG} + 4))
              f_emergencyexit ${ERRORFLAG}
              ;;
            9)
              ## Configuration error, end program gracefully.
              echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Configuration problem. Script aborted." >> ${TEMPLOG}
              ## Stop and exit the script with an error code.
              ERRORFLAG=$((${ERRORFLAG} + 8))
              f_emergencyexit ${ERRORFLAG}
              ;;
            esac
            FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'`
            if [ ${FREESPACE} -gt ${BACKUPSIZE} ]; then
              ## Enough space is now available.
              ENOUGHSPACE=1
            else
              ## Not enough space is available yet.
              ENOUGHSPACE=0
            fi
          done
        fi
    
        echo "`date +%Y-%m-%d_%H:%M:%S` --- Copying ${TEMPDIR}/${ARCHIVEFILE} to ${OFFSITEBACKDIR}/" >> ${LOGFILE}
        cp ${TEMPDIR}/${ARCHIVEFILE} ${OFFSITEBACKDIR}/${ARCHIVEFILE} 1>/dev/null 2>&1
      fi
    else
      ## Remote site is offline / unavailable.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Error: Remote site is unavailable: ${OFFSITEBACKDIR}" >> ${LOGFILE}
      ERRORFLAG=$((ERRORFLAG+16))
    fi
    
    if [ -f ${OFFSITEBACKDIR}/${ARCHIVEFILE} ]; then
      ## Remote copy worked.  Remove local archive.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Copied ${ARCHIVESIZE} bytes." >> ${LOGFILE}
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Removing ${TEMPDIR}/${ARCHIVEFILE}" >> ${LOGFILE}
      rm ${TEMPDIR}/${ARCHIVEFILE}
    
      FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'`
      ## Comment out the below line if you do not want to receive statistic emails.
      f_sendmail "Zimbra Individual Mailbox Backup" "${UCOUNT} accounts backed up.\n\nTotal archive size: $((${ARCHIVESIZE}/1024)) kb\n\nAvailable Backup Space: ${FREESPACE} kb"
    
      ## Uncomment the following 2 lines if you do not wish to have a local copy of individual mailboxes.
    #  rm ${TARGETDIR}/*.tgz
    #  rmdir ${TARGETDIR}
    else
      ## Remote copy failed.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Error copying to ${OFFSITEBACKDIR}/${ARCHIVEFILE}" >> ${LOGFILE}
      ERRORFLAG=$((ERRORFLAG+32))
    fi
    
    ## Calculate total time for backup.
    FINISHTIME="$(date +%s)"
    ELAPSEDTIME="$(expr ${FINISHTIME} - ${STARTTIME})"
    HOURS=$((${ELAPSEDTIME} / 3600))
    ELAPSEDTIME=$((${ELAPSEDTIME} - ${HOURS} * 3600))
    MINUTES=$((${ELAPSEDTIME} / 60))
    SECONDS=$((${ELAPSEDTIME} - ${MINUTES} * 60))
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Total backup time: ${HOURS} hour(s) ${MINUTES} minute(s) ${SECONDS} second(s)" >> ${LOGFILE}
    
    if [ ${ERRORFLAG} -ne 0 ]; then
      f_sendmail "Zimbra Individual Mailbox Backup Error" "${ERRORFLAG} errors detected while trying to backup ${UCOUNT} individual mailboxes."
    fi
    ## Unmount the offsite backup location.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Dismounting ${OFFSITEDIR}" >> ${LOGFILE}
    f_umount
    ## Perform cleanup routine.
    f_cleanup
    ## Exit with the combined return code value.
    echo "`date +%Y-%m-%d_%H:%M:%S` - Individual mailbox backup complete. Exit code: ${ERRORFLAG}" >> ${LOGFILE}
    exit ${ERRORFLAG}

    Sample output of mailbox-backup.log
    Code:
    2011-11-06_15:48:02 - Individual mailbox backup started.
    2011-11-06_15:48:11 --- admin@mydomain.com, 6781 bytes
    2011-11-06_15:48:13 --- lhammonds@mydomain.com, 1066 bytes
    2011-11-06_15:48:16 --- jsmith@mydomain.com, 1067 bytes
    2011-11-06_15:48:18 --- jdoe@mydomain.com, 1066 bytes
    2011-11-06_15:48:21 --- administrator@mydomain.com, 1069 bytes
    2011-11-06_15:48:23 --- foobar@mydomain.com, 1067 bytes
    2011-11-06_15:48:25 --- jvorhees@mydomain.com, 1068 bytes
    2011-11-06_15:48:28 --- conferencerooms@mydomain.com, 1067 bytes
    2011-11-06_15:48:30 --- test@mydomain.com, 1068 bytes
    2011-11-06_15:48:48 --- 9 accounts processed.
    2011-11-06_15:48:48 --- Backup time for 9 mailboxes: 0 hour(s) 0 minute(s) 46 second(s)
    2011-11-06_15:48:48 --- Setting file permissions on /var/backup/mailbox/*.tgz
    2011-11-06_15:48:48 --- Creating a single file for archiving: /var/temp/2011-11-06-15-48_mailbox-all.tar
    2011-11-06_15:48:48 --- Mounting /mnt/zmbackup
    2011-11-06_15:48:49 --- Copying /var/temp/2011-11-06-15-48_mailbox-all.tar to /mnt/zmbackup/mailbox/
    2011-11-06_15:48:49 --- Copied 61440 bytes.
    2011-11-06_15:48:49 --- Removing /var/temp/2011-11-06-15-48_mailbox-all.tar
    2011-11-06_15:48:49 --- Total backup time: 0 hour(s) 0 minute(s) 47 second(s)
    2011-11-06_15:48:49 --- Dismounting /mnt/zmbackup
    2011-11-06_15:48:49 - Individual mailbox backup complete. Exit code: 0
    Sample output of mailbox-backup-size-history.log
    Code:
    Date Archive SizeInBytes
    2011-11-06 2011-11-06-11-21_mailbox-all.tar 61440
    2011-11-06 2011-11-06-15-17_mailbox-all.tar 61440
    2011-11-06 2011-11-06-15-21_mailbox-all.tar 61440
    2011-11-06 2011-11-06-15-22_mailbox-all.tar 61440
    2011-11-06 2011-11-06-15-32_mailbox-all.tar 61440
    2011-11-06 2011-11-06-15-48_mailbox-all.tar 61440
    Sample email notification
    Code:
    Subject: Zimbra Individual Mailbox Backup
    
    9 accounts backed up.
    
    Total archive size: 60 kb
    
    Available Backup Space: 28440616 kb
    
    Server: mail
    Program: /var/scripts/prod/mailbox-backup.sh
    Log: /var/temp/mailbox-backup.log
    Sample error email notification
    Code:
    Subject: Zimbra Individual Mailbox Backup Error
    
    1 errors detected while trying to backup 9 individual mailboxes.
    
    Server: mail
    Program: /var/scripts/prod/mailbox-backup.sh
    Log: /var/temp/mailbox-backup.log
    Sample /var/log/messages
    Code:
    Nov  6 11:13:01 mail administrator: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 1
    Nov  6 11:15:02 mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 2
    Nov  6 11:18:03 mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 4
    Nov  6 11:23:04 mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 8
    Nov  6 11:33:05 mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 16
    Nov  6 11:43:06 mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 32
    Last edited by LHammonds; 01-16-2012 at 02:47 PM.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  2. #12
    LHammonds's Avatar
    LHammonds is offline Special Member
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    3

    Default Zimbra Restore

    Zimbra Restore

    I see the need to restore from at least several different scenarios. That means coming up with a restore plan and scripts to help speed up restore time and reduce errors during emergency scenarios. This also means testing each of these restore plans; otherwise you are playing Russian roulette with more than one bullet in the revolver!

    Here are the scenarios I am going to cover and test:


    1. Restore individual mailbox from a local copy - we will restore a mailbox account using the latest individual mailbox backup which is stored on the local server. This can be done at any time and will not impact server uptime.
    2. Restore individual mailbox from an offsite location - we will restore a mailbox account using one of the archives located on the offsite storage. This can be done at any time and will not impact server uptime.
    3. Restore existing server from a local copy (fastest, but potentially problematic) - we will use the local folder used for doing daily rsync backups. The caveat here is that you only want to do this if the last rsync was from the offline rsync script...meaning it is a complete snapshot and not from any of the online rsyncs which did not copy open and in-use files. This will cause the server to be offline, so be sure to coordinate and plan accordingly.
    4. Restore existing server from an offsite location (slower, but more reliable) - we will restore the server using one of the archives located on the offsite storage. This will cause the server to be offline, so be sure to coordinate and plan accordingly.
    5. Restore to new server (production server unavailable) - we will assume the production server is offline and inaccessible. We will configure a new server and restore using one of the archives located on the offsite storage.


    ** UNFINISHED **
    Last edited by LHammonds; 11-06-2011 at 04:23 PM.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  3. #13
    LHammonds's Avatar
    LHammonds is offline Special Member
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    3

    Smile Restore individual mailbox from a local copy

    Scenario #1 Restore individual mailbox from a local copy

    This process will assume you are wanting to restore a mailbox back to the way it was when last backed up. This means anything new since the backup will be wiped out. If this is not desired, a simple solution (which will be my standard protocol) will be to create a new mailbox and restore to it.

    If creating a new mailbox, be sure to set a password on it and test it by logging into the account...this also creates the mailbox store when accessed the 1st time. Then copy the user's mailbox archive to look like the archive for the restore account.

    Example #1 - Restore the admin mailbox.

    Code:
    /var/scripts/prod/mailbox-restore-local.sh admin@mydomain.com
    Onscreen result:
    Code:
    2011-11-06_17:45:55 - Restored mailbox --> admin@mydomain.com
    Recorded event in the log: /var/temp/mailbox-restore.log
    Code:
    2011-11-06_17:45:55 - Restored mailbox --> admin@mydomain.com
    Example #2 - Restore the admin mailbox to a temporary account called "restore".

    1. Login to the admin console: h t t p s ://mail.mydomain.com:7071
    2. Create a new account called restore and set the password
    3. Login to the webmail using the restore account: h t t p s ://mail.mydomain.com
    4. Once logged in to verify the empty account works, logout
    5. Access the Ubuntu server. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass)
    6. Type the following:
      Code:
      
      cp /var/backup/mailbox/admin@mydomain.com.tgz /var/backup/mailbox/restore@mydomain.com.tgz
      /var/scripts/prod/mailbox-restore-local.sh restore@mydomain.com
      


    /var/scripts/prod/mailbox-restore-local.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : mailbox-restore-local.sh
    ## Version       : 1.0
    ## Date          : 2011-11-06
    ## Author        : LHammonds
    ## Purpose       : Restore individual mailbox accounts.
    ## Compatibility : Verified on Ubuntu Server 10.04.3 LTS, Zimbra 7.1.2 - 7.1.3 OSE
    ## Requirements  : Zimbra must be online, must be run as root user.
    ## Run Frequency : Manual as needed.
    ## Exit Codes    : (if multiple errors, value is the addition of codes)
    ##    0 = success
    ##    1 = parameter failure
    ##    2 = unknown mailbox account
    ##    4 = missing archive folder
    ##    8 = missing archive file
    ##   16 = mailbox restore failure
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-10-24 LTH Created script.
    ## 2011-11-06 LTH Move common variables and functions to external file.
    #############################################
    
    ## Import common variables and functions.
    source /var/scripts/common/standard.conf
    
    ## Define local variables.
    LOGFILE="${TEMPDIR}/mailbox-restore.log"
    ARCHIVEDIR="${BACKUPDIR}/mailbox"
    ACCT=$1
    RETURNVALUE=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    
    function f_showhelp()
    {
      echo ""
      echo "Usage : ${SCRIPTNAME} user@${MYDOMAIN}"
      echo ""
      echo "NOTE: You could create a temporary account and copy the desired archive"
      echo "      to be named the same as the temporary account in order to restore"
      echo "      to a new mailbox and preserve the existing mailbox."
      echo "      Example:"
      echo "        cd ${ARCHIVEDIR}"
      echo "        cp admin@${MYDOMAIN}.tgz restore@${MYDOMAIN}.tgz"
      echo "        ${SCRIPTNAME} restore@${MYDOMAIN}"
      echo ""
    }
    
    function f_emergencyexit()
    {
      ## Purpose: Exit script as cleanly as possible.
      ## Parameter #1 = Error Code.
      echo -e "Individual mailbox restore aborted.  EXIT CODE: ${1}\n"
      exit $1
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    
    ## Check command-line parameters.
    case "$1" in
      "")
        f_showhelp
        f_emergencyexit 1
        ;;
      --help|-h|-?)
        f_showhelp
        f_emergencyexit 1
        ;;
      *)
        ACCT=$1
        ;;
    esac
    
    ## Check to make sure the archive folder exists.
    if [ ! -d "${ARCHIVEDIR}" ]; then
      ## ERROR: Archive folder does not exist.
      echo "ERROR: The archive folder does not exist!  ${ARCHIVEDIR}"
      f_emergencyexit 4
    fi
    
    ## Check to make sure the archive exists.
    if [ ! -f "${ARCHIVEDIR}/${ACCT}.tgz" ]; then
      ## ERROR: Archive does not exist.
      echo "ERROR: The archive does not exist!   ${ARCHIVEDIR}/${ACCT}.tgz"
      f_emergencyexit 8
    fi
    
    ## Check to make sure parameter is a valid mailbox account name.
    if `su - zimbra -c "zmprov -l gaa" | grep -q ${ACCT}`
    then
      ## The parameter contains a valid mailbox account.
      echo "" > /dev/null
    else
      ## Invalid mailbox account.
      echo "ERROR: Invalid mailbox: ${ACCT}"
      f_showhelp
      f_emergencyexit 2
    fi
    
    ${ZIMBRADIR}/bin/zmmailbox -z -m ${ACCT} postRestURL "//?fmt=tgz&resolve=reset" ${ARCHIVEDIR}/${ACCT}.tgz
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## Something went wrong.
      echo "`date +%Y-%m-%d_%H:%M:%S` - Error restoring ${ACCT}, RETURN VALUE = ${RETURNVALUE}" | tee -a ${LOGFILE}
      f_emergencyexit 16
    fi
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restored mailbox --> ${ACCT}" | tee -a ${LOGFILE}
    f_sendmail "Zimbra User Mailbox Restore" "${ACCT} was manually restored."
    
    exit 0
    If you do not supply an account to the script as a parameter, it will display the following help text:
    Code:
    /var/scripts/prod/mailbox-restore.sh
    
    Usage : ./mailbox-restore-local.sh user@mydomain.com
    
    NOTE: You could create a temporary account and copy the desired archive
          to be named the same as the temporary account in order to restore
          to a new mailbox and preserve the existing mailbox.
          Example:
            cd /var/backup/mailbox
            cp admin@mydomain.com.tgz restore@mydomain.com.tgz
            /var/scripts/prod/mailbox-restore.sh restore@mydomain.com
    
    Individual mailbox restore aborted.  EXIT CODE: 1
    NOTE: The above will actually use your domain in all examples because it pulls the domain from the standard.conf file.
    Last edited by LHammonds; 11-10-2011 at 05:03 PM.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  4. #14
    LHammonds's Avatar
    LHammonds is offline Special Member
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    3

    Smile Restore individual mailbox from an offsite location

    Scenario #2 Restore individual mailbox from an offsite location

    Here is the general outline of the script:

    Parameter #1 (required) = Account in which we will find the archive for.
    Parameter #2 (optional) = Account in which we will restore the selected archive into (default = parameter 1)

    Step 1 - Display all offsite archives (order oldest to newest) and present list to user for selection
    Step 2 - Extract only the account's file from the archive to the temp folder
    Step 3 - Restore archive to desired account

    ----------------------

    /var/scripts/prod/mailbox-restore.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : mailbox-restore.sh
    ## Version       : 1.0
    ## Date          : 2011-11-08
    ## Author        : LHammonds
    ## Purpose       : Restore individual mailbox accounts.
    ## Compatibility : Verified on Ubuntu Server 10.04.3 LTS, Zimbra 7.1.3 OSE
    ## Requirements  : Must be run as root user
    ## Run Frequency : Manual as needed.
    ## Parameters    :
    ##    1 = (Required) Account to search for archive (e.g. jdoe@mydomain.com)
    ##    2 = (Optional) Account to restore archive to (e.g. restore@mydomain.com)
    ## Exit Codes    :
    ##    0 = Success or non-critical problem
    ##    1 = Archive folder does not exist
    ##    2 = Invalid mailbox account
    ##    3 = Mailbox restore failure
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-11-08 LTH Created script.
    #############################################
    
    ## Import common variables and functions.
    source /var/scripts/common/standard.conf
    
    ## Define local variables.
    LOGFILE="${TEMPDIR}/mailbox-restore.log"
    ARCHIVEDIR="${OFFSITEDIR}/mailbox"
    ACCT1=$1
    ACCT2=$2
    RETURNVALUE=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    
    function f_showhelp()
    {
      echo ""
      echo "Usage : ${SCRIPTNAME} user@${MYDOMAIN} restore@${MYDOMAIN} (optional)"
      echo ""
      echo "NOTE: If you specify a 2nd account, the 1st account data will be"
      echo "      restored to the 2nd account."
      echo ""
    }
    
    function f_emergencyexit()
    {
      ## Purpose: Exit script as cleanly as possible.
      ## Parameter #1 = Error Code.
    
      if [ -f ${REMOTEDIR}/online.txt ]; then
        ## Dismount the remote site.
        f_umount
      fi
      echo -e "Individual mailbox restore aborted.  EXIT CODE: ${1}\n"
      exit $1
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    
    ## Check command-line parameters.
    case "$1" in
      "")
        f_showhelp
        exit 0
        ;;
      --help|-h|-?)
        f_showhelp
        exit 0
        ;;
      *)
        ACCT1=$1
        ACCT2=$2
        ;;
    esac
    
    if [ "${ACCT1}" = "" ]; then
      ## Required parameter not specified.
      f_showhelp
      exit 0
    fi
    
    if [ "${ACCT2}" = "" ]; then
      ## Restore to the same account.
      ACCT2=${ACCT1}
    fi
    
    ## Mount the remote site.
    f_mount
    
    ## Check to make sure the archive folder exists.
    if [ ! -d "${ARCHIVEDIR}" ]; then
      ## ERROR: Archive folder does not exist.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: The archive folder does not exist!  ${ARCHIVEDIR}, EC=1" | tee -a ${LOGFILE}
      f_emergencyexit 1
    fi
    
    ## Define user prompt using the special PS3 variable.
    PS3="Type number for the desired archive or 'q' to quit: "
    
    ## Get sorted list of all archives (newest at the bottom).
    FILELIST=$(find ${ARCHIVEDIR}/*.tar -maxdepth 1 -type f | sort -f)
    
    ## Prompt user to select a file to use.
    ## NOTE: If it is a long list, user can scroll up if
    ##       using PuTTY to see older files.
    select GETFILE in ${FILELIST}; do
      if [ "${GETFILE}" != "" ]; then
        FILENAME=${GETFILE}
      fi
      break
    done
    
    if [ "${FILENAME}" = "" ]; then
      ## User opted to quit.
      echo -e "Exiting restore program.\n"
      exit 0
    fi
    
    ## Extract just the mailbox file from the archive.
    tar --extract -C ${TEMPDIR} --file=${FILENAME} --wildcards "*${ACCT1}*" --no-anchored --strip-components=3
    
    if [ ! -f ${TEMPDIR}/${ACCT1}.tgz ]; then
      ## File did not exist in the archive.
      echo -e "No file found matching ${ACCT1}\n"
      exit 0
    fi
    
    ## Check to make sure parameter is a valid mailbox account name.
    if `su - zimbra -c "zmprov -l gaa" | grep -q ${ACCT2}`
    then
      ## The parameter contains a valid mailbox account.
      echo "" > /dev/null
    else
      ## Invalid mailbox account.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Invalid mailbox: ${ACCT2}, EC=2" | tee -a ${LOGFILE}
      f_showhelp
      f_emergencyexit 2
    fi
    
    ${ZIMBRADIR}/bin/zmmailbox -z -m ${ACCT2} postRestURL "//?fmt=tgz&resolve=reset" ${TEMPDIR}/${ACCT1}.tgz
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## Something went wrong.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Restore failure on ${ACCT2}, RETURN VALUE=${RETURNVALUE}, EC=3" | tee -a ${LOGFILE}
      f_emergencyexit 3
    fi
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restored mailbox --> ${ACCT2}" | tee -a ${LOGFILE}
    f_sendmail "Zimbra User Mailbox Restore" "${ACCT2} was manually restored from ${ACCT1}"
    
    ## Clean up temporary files.
    if [ -f ${TEMPDIR}/${ACCT1}.tgz ]; then
      rm ${TEMPDIR}/${ACCT1}.tgz
    fi
    
    ## Dismount the remote site.
    f_umount
    exit 0
    /var/temp/mailbox-restore.log (sample)
    Code:
    2011-11-08_13:01:39 - ERROR: The archive folder does not exist!  /mnt/zmbackup/configtypo, EC=1
    2011-11-08_15:54:34 - ERROR: Restore failure for admin@mydomain.com, RETURN VALUE=2, EC=3
    2011-11-08_15:55:47 - Restored mailbox --> admin@mydomain.com
    2011-11-08_15:57:50 - Restored mailbox --> lhammonds@mydomain.com
    2011-11-08_16:00:14 - Restored mailbox --> ddiggler@mydomain.com
    2011-11-08_16:01:16 - Restored mailbox --> restore@mydomain.com
    2011-11-08_16:28:38 - ERROR: Invalid mailbox: bogus@mydomain.com, EC=2
    Last edited by LHammonds; 11-10-2011 at 05:04 PM.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  5. #15
    LHammonds's Avatar
    LHammonds is offline Special Member
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    3

    Smile Restore existing server from a local copy

    Scenario #3 Restore existing server from a local copy

    We will use the local folder used for doing daily rsync backups.

    The caveot here is that you only want to do this if the last rsync was from the offline rsync script...meaning it is a complete snapshot and not from any of the online rsyncs which did not copy open and in-use files. If the last rsync was the online version, we will need to use a different script to access remote storage.

    This script is intended to be run manually as the root user.

    Here are the instructions to restore the currently running server using the script below:


    1. Before running this, make sure everyone is aware that Zimbra will be going down to be restored.
    2. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass)
    3. Run the script by typing /var/scripts/prod/restore-rsync-local.sh
    4. If you missed any of the output on the screen, you can view the log by typing vi /var/temp/restore-rsync.log


    restore-rsync-local.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : restore-rsync-local.sh
    ## Version       : 1.1
    ## Date          : 2011-11-08
    ## Author        : LHammonds
    ## Purpose       : Restore Zimbra using rsync and the local backup.
    ## Compatibility : Verifed on Ubuntu Server 10.04.3 LTS, Zimbra 7.1.3 OSE
    ## Requirements  : Must be run as root user, last rsync should be the offline version.
    ## Run Frequency : Manual as needed.
    ## Exit Codes    : Errors displayed to screen.
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-10-18 LTH Created script.
    ## 2011-10-31 LTH Added better logging, error checks, user input.
    ## 2011-11-08 LTH Move common variables and funtions to external file.
    #############################################
    
    ## Import common variables and functions. ##
    source /var/scripts/common/standard.conf
    
    LOGFILE="${TEMPDIR}/restore-rsync.log"
    LOCKFILE="${TEMPDIR}/rsync.lock"
    RSYNCLOGFILE="${TEMPDIR}/rsync.log"
    PRODROOT="/opt"
    OLDDIR="${BACKUPDIR}/zimbra.old"
    LOCALBACKUP="${BACKUPDIR}/zimbra/zimbra"
    LOCALBACKUPDEV="/dev/mapper/LVG-bak"
    NOBACKUP=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    
    function f_cleanup()
    {
      if [ -f ${LOCKFILE} ];then
        ## Remove lock file so other rsync jobs can run.
        rm ${LOCKFILE} 1>/dev/null 2>&1
      fi
    }
    
    function f_checkspace()
    {
      ## Need to check and see if there is enough space available to move Zimbra to a backup location.
      FREESPACE=`df -k ${LOCALBACKUPDEV} | grep ${LOCALBACKUPDEV} | awk '{ print $4 }'`
      BACKUPSIZE=`du -sk ${ZIMBRADIR} | awk '{ print $1 }'`
    
      if [ ${BACKUPSIZE} -gt ${FREESPACE} ]; then
        echo "There is not enough freespace available to archive production Zimbra."
        echo "Zimbra size is ${BACKUPSIZE} kb, Free space is ${FREESPACE} kb"
        echo ""
        echo "To override and not make a backup 1st, use the --nobackup option."
        echo ""
        echo "Example:  ${SCRIPTNAME} --nobackup"
        echo ""
        f_cleanup
        exit 1
      fi
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    
    if [ -f ${LOCKFILE} ]; then
      # rsync lock file detected.  Abort script.
      echo "Restore rsync aborted"
      echo "This script tried to run but detected the lock file: ${LOCKFILE}"
      echo "Please check to make sure the file does not remain when rsync is not actually running."
      f_sendmail "restore rsync aborted" "This script tried to run but detected the lock file: ${LOCKFILE}\n\nPlease check to make sure the file does not remain when rsync is not actually running."
      exit 1
    else
      echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE}
    fi
    
    clear
    echo ""
    
    # Determine if we are going to backup existing folder or not.
    if [ "$1" = "--nobackup" ]; then
      NOBACKUP=1
    else
      NOBACKUP=0
      f_checkspace
    fi
    
    ## Requirement Check: Script must run as root user.
    if [ "$(id -u)" != "0" ]; then
      ## FATAL ERROR DETECTED: Document problem and terminate script.
      echo "ERROR: Root user required for restore."
      echo ""
      f_cleanup
      exit 1
    fi
    
    if [ ! -f ${RSYNCLOGFILE} ]; then
      ## FATAL ERROR DETECTED: Document problem and terminate script.
      echo "ERROR: Cannot find log: ${RSYNCLOGFILE}"
      echo "This log file is required to determine status of last rsync job."
      echo ""
      f_cleanup
      exit 1
    fi
    
    
    echo "This restore method can only work if the last rsync backup was"
    echo "the 'Offline' version which allows for a complete backup."
    echo ""
    echo "The 'Online' version is only a partial backup and will not work."
    echo ""
    read -p "The next step will try to auto-detect the last backup. Continue (y/n)? "
    if [ "${REPLY}" != "y" ]; then
      echo "Script cancelled."
      echo ""
      f_cleanup
      exit 1
    fi
    echo ""
    
    tail -n1 ${RSYNCLOGFILE} | grep -q "Offline RSync backup exit code"
    if [ $? -eq 0 ]; then
      echo "Last rsync seems to have been a good offline backup"
    else
      echo "WARNING: This script did not see the offline entry at the end of the log file."
    fi
    echo ""
    
    echo "--------------------------------------------------------"
    tail -n5 ${RSYNCLOGFILE}
    echo "--------------------------------------------------------"
    echo ""
    
    read -p "Do you see 'Offline RSync backup exit code: 0' in the log above (y/n)? "
    if [ "${REPLY}" != "y" ]; then
      echo "Script cancelled."
      echo ""
      f_cleanup
      exit 1
    fi
    
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restore started." | tee -a ${LOGFILE}
    
    StartTime="$(date +%s)"
    
    ## Stop Zimbra Services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Stopping Zimbra services." | tee -a ${LOGFILE}
    /etc/init.d/zimbra stop
    
    ## Kill any orphaned Zimbra processes.
    pkill -9 -u zimbra 1>/dev/null 2>&1
    
    if [ ${NOBACKUP} -eq 1 ]; then
      ## Leave current production folder alone.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- 'No Backup' override specified, skipping folder move." | tee -a ${LOGFILE}
    else
      ## Move production folder to backup location.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Moving Zimbra from ${ZIMBRADIR} to ${OLDDIR}." | tee -a ${LOGFILE}
      mv ${ZIMBRADIR} ${OLDDIR}
      RETURNVALUE=$?
      if [ ${RETURNVALUE} -ne 0 ]; then
        ## Could not move Zimbra to backup location.
        ## Document error and terminate script.
        echo ""
        echo "ERROR: The move was not successful. Return value = ${RETURNVALUE}" | tee -a ${LOGFILE}
        echo ""
        echo "COMMAND was 'mv ${ZIMBRADIR} ${OLDDIR}'" | tee -a ${LOGFILE}
        echo ""
        echo "The Zimbra services are currently stopped and an unknown amount of files were moved"
        echo "from the source.  That means there some files are still in ${ZIMBRADIR}"
        echo "and some files were moved to ${OLDDIR}"
        echo "This needs to be resolved before you can start the services back up."
        echo "Either move the files back or delete all files and run the restore."
        f_cleanup
        exit 99
      fi
    fi
    
    ## Now that Zimbra is offline, rsync can restore all files.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Restoring files with rsync." | tee -a ${LOGFILE}
    rsync -apogHK ${LOCALBACKUP} ${PRODROOT}
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## rsync command failed.  Display warning message.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: rsync return value = {$RETURNVALUE}" | tee -a ${LOGFILE}
    fi
    
    if [ ! -f ${ZIMBRADIR}/.install_history ]; then
      ## Something is not right...at least this file should exist here.
      ## NOTE: This is not a guarentee all files were restored, it is just a quick check.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Missing file(s).  Did not find ${ZIMBRADIR}/.install_history" | tee -a ${LOGFILE}
    fi
    
    ## Fix permissions.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Fixing file ownership and permissions." | tee -a ${LOGFILE}
    chown -R zimbra:zimbra ${ZIMBRADIR}
    ${ZIMBRADIR}/libexec/zmfixperms
    
    ## Start Zimbra services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Starting Zimbra services." | tee -a ${LOGFILE}
    /etc/init.d/zimbra start
    
    ## Calculates downtime of Zimbra services.
    FinishTime="$(date +%s)"
    ElapsedTime="$(expr ${FinishTime} - ${StartTime})"
    Hours=$((${ElapsedTime} / 3600))
    ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600))
    Minutes=$((${ElapsedTime} / 60))
    Seconds=$((${ElapsedTime} - ${Minutes} * 60))
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Zimbra restore time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" | tee -a ${LOGFILE}
    
    ## Show status of Zimbra services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Status of Zimbra services:" | tee -a ${LOGFILE}
    su - zimbra -c "zmcontrol status" | tee -a ${LOGFILE}
    
    if [ -d ${OLDDIR} ]; then
      echo -e "\n\nDon't forget to purge ${OLDDIR} once everything is running smoothly again.\n\n"
    fi
    
    f_cleanup
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restore completed." | tee -a ${LOGFILE}
    Sample output from script:
    Code:
    2011-10-18_16:26:40 - Restore started.
    2011-10-18_16:26:40 --- Stopping Zimbra services.
    Host mail.mydomain.com
            Stopping stats...Done.
            Stopping mta...Done.
            Stopping spell...Done.
            Stopping snmp...Done.
            Stopping cbpolicyd...Done.
            Stopping archiving...Done.
            Stopping antivirus...Done.
            Stopping antispam...Done.
            Stopping imapproxy...Done.
            Stopping memcached...Done.
            Stopping mailbox...Done.
            Stopping logger...Done.
            Stopping zmconfigd...Done.
            Stopping ldap...Done.
    2011-10-18_16:27:22 --- Moving Zimbra from /opt/zimbra to /temp/zimbra.old.
    2011-10-18_16:27:22 --- Restoring files with rsync.
    2011-10-18_16:31:53 --- Fixing file ownership and permissions.
    2011-10-18_16:31:55 --- Starting Zimbra services.
    Host mail.mydomain.com
            Starting ldap...Done.
            Starting zmconfigd...Done.
            Starting logger...Done.
            Starting mailbox...Done.
            Starting antispam...Done.
            Starting antivirus...Done.
            Starting snmp...Done.
            Starting spell...Done.
            Starting mta...Done.
            Starting stats...Done.
    2011-10-18_16:33:25 --- Zimbra restore time: 0 hour(s) 6 minute(s) 45 second(s)
    2011-10-18_16:33:25 --- Status of Zimbra services:
    Host mail.mydomain.com
            antispam                Running
            antivirus               Running
            ldap                    Running
            logger                  Running
            mailbox                 Running
            mta                     Running
            snmp                    Running
            spell                   Running
            stats                   Running
            zmconfigd               Running
    2011-10-18_16:33:55 - Restore completed.
    Last edited by LHammonds; 01-30-2012 at 03:08 PM.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  6. #16
    LHammonds's Avatar
    LHammonds is offline Special Member
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    3

    Smile Restore existing server from an offsite location

    Scenario #4 Restore existing server from an offsite location

    /var/scripts/prod/rsync-restore.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : rsync-restore.sh
    ## Version       : 1.0
    ## Date          : 2011-11-08
    ## Author        : LHammonds
    ## Purpose       : Restore Zimbra using rsync from a remote archive.
    ## Compatibility : Verified on Ubuntu Server 10.04.3 LTS, Zimbra 7.1.2 - 7.1.3 OSE
    ## Requirements  : Must be root user, p7zip-full (if ARCHIVEMETHOD=tar.7z)
    ## Run Frequency : Manual as needed.
    ## Exit Codes    :
    ##    0 = Success or non-critical problem
    ##    1 = Archive folder does not exist
    ##    2 = Archive extration failed
    ##    3 = Zimbra restore failure
    ##   99 = Move production to temp directory failure (bad)
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-11-08 LTH Created script.
    #############################################
    
    ## Import common variables and functions. ##
    source /var/scripts/common/standard.conf
    
    LOGFILE="${TEMPDIR}/rsync-restore.log"
    LOCKFILE="${TEMPDIR}/rsync.lock"
    ARCHIVEDIR="${OFFSITEDIR}/zimbra"
    OLDDIR="${BACKUPDIR}/zimbra.old"
    NOBACKUP=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    
    function f_cleanup()
    {
      if [ -f ${LOCKFILE} ];then
        ## Remove lock file so other rsync jobs can run.
        rm ${LOCKFILE} 1>/dev/null 2>&1
      fi
    }
    
    function f_checkspace()
    {
      ## Need to check and see if there is enough space available to move Zimbra to a backup location.
      FREESPACE=`df -k ${BACKUPDIR} | grep ${BACKUPDIR} | awk '{ print $4 }'`
      BACKUPSIZE=`du -sk ${ZIMBRADIR} | awk '{ print $1 }'`
    
      if [ ${BACKUPSIZE} -gt ${FREESPACE} ]; then
        echo "There is not enough freespace available to archive production Zimbra."
        echo "Zimbra size is ${BACKUPSIZE} kb, Free space is ${FREESPACE} kb"
        echo ""
        echo "To override and not make a backup 1st, use the --nobackup option."
        echo ""
        echo "Example:  ${SCRIPTNAME} --nobackup"
        echo ""
        f_cleanup
        exit 1
      fi
    }
    
    function f_emergencyexit()
    {
      ## Purpose: Exit script as cleanly as possible.
      ## Parameter #1 = Error Code.
    
      if [ -f ${REMOTEDIR}/online.txt ]; then
        ## Dismount the remote site.
        f_umount
      fi
      echo -e "Zimbra restore aborted.  EXIT CODE: ${1}\n"
      exit $1
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    
    if [ -f ${LOCKFILE} ]; then
      ## Restore lock file detected.  Abort script.
      echo "Restore aborted"
      echo "This script tried to run but detected the lock file: ${LOCKFILE}"
      echo "Please check to make sure the file does not remain when rsync or restore is not actually running."
      f_sendmail "Zimbra Restore aborted" "This script tried to run but detected the lock file: ${LOCKFILE}\n\nPlease check to make sure the file does not remain when rsync/restore is not actually running."
      exit 1
    else
      echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE}
    fi
    
    clear
    echo ""
    
    ## Determine if we are going to backup existing folder or not.
    if [ "$1" = "--nobackup" ]; then
      NOBACKUP=1
    else
      NOBACKUP=0
      ## 
      f_checkspace
    fi
    
    ## Requirement Check: Script must run as root user.
    if [ "$(id -u)" != "0" ]; then
      ## FATAL ERROR DETECTED: Document problem and terminate script.
      echo "ERROR: Root user required for restore."
      echo ""
      f_cleanup
      exit 1
    fi
    
    ## Mount the remote site.
    f_mount
    
    ## Check to make sure the archive folder exists.
    if [ ! -d "${ARCHIVEDIR}" ]; then
      ## ERROR: Archive folder does not exist.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: The archive folder does not exist!  ${ARCHIVEDIR}, EC=1" | tee -a ${LOGFILE}
      f_cleanup
      f_emergencyexit 1
    fi
    
    ## Define user prompt using the special PS3 variable.
    PS3="Type number for the desired archive or 'q' to quit: "
    
    echo "                Z I M B R A   R E S T O R E"
    echo "                ---------------------------"
    ## Get sorted list of all archives (newest at the bottom).
    FILELIST=$(find ${ARCHIVEDIR}/*.${ARCHIVEMETHOD} -maxdepth 1 -type f | sort -f)
    
    ## Prompt user to select a file to use.
    ## NOTE: If it is a long list, user can scroll up if
    ##       using PuTTY to see older files.
    select GETFILE in ${FILELIST}; do
      if [ "${GETFILE}" != "" ]; then
        FILENAME=${GETFILE}
      fi
      break
    done
    
    if [ "${FILENAME}" = "" ]; then
      ## User opted to quit.
      echo -e "Exiting restore program.\n"
      f_cleanup
      f_umount
      exit 0
    fi
    
    echo -e "\nSelected file: ${FILENAME}\n"
    echo -e "   This restore will shutdown Zimbra and make it"
    echo -e "   unavailable while it is being restored.\n"
    read -p "Are you absolutely sure you wish to restore (y/n)? "
    
    if [ "${REPLY}" != "y" ]; then
      echo -e "\n\nRestore aborted.\n\n"
      f_cleanup
      exit 0
    fi
    
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restore started." | tee -a ${LOGFILE}
    
    StartTime="$(date +%s)"
    
    ## Stop Zimbra Services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Stopping Zimbra services." | tee -a ${LOGFILE}
    /etc/init.d/zimbra stop
    
    ## Kill any orphaned Zimbra processes.
    pkill -9 -u zimbra 1>/dev/null 2>&1
    
    if [ ${NOBACKUP} -eq 1 ]; then
      ## Leave current production folder alone.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- 'No Backup' override specified, skipping folder move." | tee -a ${LOGFILE}
    else
      ## Move production folder to backup location.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Moving Zimbra from ${ZIMBRADIR} to ${OLDDIR}." | tee -a ${LOGFILE}
      mv ${ZIMBRADIR} ${OLDDIR}
      RETURNVALUE=$?
      if [ ${RETURNVALUE} -ne 0 ]; then
        ## Could not move Zimbra to backup location.
        ## Document error and terminate script.
        echo ""
        echo "ERROR: The move was not successful. Return value = ${RETURNVALUE}" | tee -a ${LOGFILE}
        echo ""
        echo "COMMAND was 'mv ${ZIMBRADIR} ${OLDDIR}'" | tee -a ${LOGFILE}
        echo ""
        echo "The Zimbra services are currently stopped and an unknown amount of files were moved"
        echo "from the source.  That means there some files are still in ${ZIMBRADIR}"
        echo "and some files were moved to ${OLDDIR}"
        echo "This needs to be resolved before you can start the services back up."
        echo "Either move the files back or delete all files and run the restore."
        f_cleanup
        exit 99
      fi
    fi
    
    ## Now that Zimbra is offline, we can restore all files.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Restoring files from ${FILENAME}" | tee -a ${LOGFILE}
    
    if [ ! -d ${ZIMBRADIR} ]; then
      mkdir -p ${ZIMBRADIR}
    fi
    
    ## Extract the entire contents of the archive back to its original location.
    case "${ARCHIVEMETHOD}" in
    tar.7z)
      7za x -so -w/${TEMPDIR} ${FILENAME} | tar -C ${ZIMBRADIR} --extract --strip-components=4 --file -
      ;;
    tgz)
      tar --extract -C ${ZIMBRADIR} --strip-components=4 --file=${FILENAME}
      ;;
    *)
      tar --extract -C ${ZIMBRADIR} --strip-components=4 --file=${FILENAME}
      ;;
    esac
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## tar extract command failed.  Display warning message.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: tar extract return value = {$RETURNVALUE}" | tee -a ${LOGFILE}
      f_cleanup
      f_emergencyexit 2
    fi
    
    if [ ! -f ${ZIMBRADIR}/.install_history ]; then
      ## Something is not right...at least this file should exist here.
      ## NOTE: This is not a guarentee all files were restored, it is just a quick check.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Missing file(s).  Did not find ${ZIMBRADIR}/.install_history" | tee -a ${LOGFILE}
    fi
    
    ## Fix permissions.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Fixing file ownership and permissions." | tee -a ${LOGFILE}
    chown -R zimbra:zimbra ${ZIMBRADIR}
    ${ZIMBRADIR}/libexec/zmfixperms
    
    ## Start Zimbra services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Starting Zimbra services." | tee -a ${LOGFILE}
    /etc/init.d/zimbra start
    
    ## Calculates downtime of Zimbra services.
    FinishTime="$(date +%s)"
    ElapsedTime="$(expr ${FinishTime} - ${StartTime})"
    Hours=$((${ElapsedTime} / 3600))
    ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600))
    Minutes=$((${ElapsedTime} / 60))
    Seconds=$((${ElapsedTime} - ${Minutes} * 60))
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Zimbra restore time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" | tee -a ${LOGFILE}
    
    ## Show status of Zimbra services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Status of Zimbra services:" | tee -a ${LOGFILE}
    su - zimbra -c "zmcontrol status" | tee -a ${LOGFILE}
    
    if [ -d ${OLDDIR} ]; then
      echo -e "\n\nDon't forget to purge ${OLDDIR} once everything is running smoothly again.\n\n"
    fi
    
    f_cleanup
    ## Dismount the remote site.
    f_umount
    
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restore completed." | tee -a ${LOGFILE}
    Last edited by LHammonds; 11-10-2011 at 05:07 PM.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  7. #17
    LHammonds's Avatar
    LHammonds is offline Special Member
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    3

    Smile Restore to new server

    Scenario #5 Restore to new server

    Not enough time to test and document this scenario.

    I'm fairly confident that I can make this work so I'll postpone this item for now.

    ** UNFINISHED **
    Last edited by LHammonds; 11-10-2011 at 05:09 PM.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  8. #18
    LHammonds's Avatar
    LHammonds is offline Special Member
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    3

    Default I am not a very good salesman

    The restore scripts have now been posted and all prior scripts in this thread have been slightly modified and standardized based on the latest scripts.

    I'm going to pause on the remaining automation scripts I have planned and will now focus purely on the aspects of "selling" this package as a viable option for replacing our Exchange server...which includes preparing the initial tutorials on how to access and use the new mail server. They will be geared towards my users which are all Outlook web, Outlook client and iPhone users.

    Once those areas are at an acceptable point, I'll then punt this project over to my boss for him to take it up the ladder. While he is doing that, I will then test and document how to convert all our existing data over to the new server.

    LHammonds
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  9. #19
    LHammonds's Avatar
    LHammonds is offline Special Member
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    3

    Default Script correction for Crontab problem

    I have updated the /var/scripts/common/standard.conf file to resolve a problem I noticed once I let the server run fully automated and hands off.

    None of the scripts were sending me notification emails.

    I would manually run mailbox-backup.sh and it would send the email I was expecting but in crontab, it would not.

    All the programs were working perfectly and doing exactly what they were supposed to do. However, the "sendmail" simply failed to do anything.

    Research suggested it was due to there not being a path environment in crontab, however, the tar, 7z and rsync programs worked without requiring a full path but sendmail (located in /usr/sbin) simply would not function. A member sent me a PM and let me know heirloom sendmail was not necessary to be installed...so I removed it from my documentation and will then just use the sendmail program that comes with Zimbra...located in /opt/zimbra/postfix/sbin/sendmail

    So I edited the common file that includes sendmail function as follows:

    original:
    Code:
      sendmail -t < ${MAILFILE}
    modified:
    Code:
     ${ZIMBRADIR}/postfix/sbin/sendmail -t < ${MAILFILE}
    Now the scheduled programs are able to send the mail as expected.

    I probably need to re-configure the root crontab to include environment settings. I need to research that a bit more to avoid things like this in the future.

    The "whereis" command is helpful for determining the location of the utilities used. Here are the locations of the utilities used in my scripts:

    /bin/cp
    /bin/df
    /bin/rm
    /bin/mv
    /bin/su
    /bin/sed
    /bin/tar
    /bin/echo
    /bin/mkdir
    /bin/chmod
    /bin/touch
    /bin/mount
    /usr/bin/7z
    /usr/bin/awk
    /usr/bin/rsync
    /usr/bin/logger
    /usr/sbin/sendmail

    So the path to the files above can be reduced to just this:

    /bin
    /usr/bin
    /usr/sbin

    Sendmail was the ONLY utility located in the /usr/sbin path so it is safe to say that my problem was due to not having /usr/sbin in the environment search path. All other utilities located in /bin and /usr/bin worked without requiring the full path so those were apparently set in the path. But since I only call that program in the "common" include script, I only needed to modify that one file and specify the entire path in order to make it work for everything.

    The other solution is to include the sbin in the path...which I don't know how to do at the moment.

    EDIT

    I would not advise anyone to ever "edit" a live crontab schedule by typing "crontab -e" but rather edit a saved schedule file and then load the schedule file. This will allow you to make backups of the schedule so you can always go back to a known-good schedule or at least back to the way it was before you made a change...assuming you always work with a copy of the schedule 1st.

    Here is my root crontab scheduling file:

    /var/scripts/prod/crontab.root
    Code:
    ########################################
    # Name: Crontab Schedule for root user
    # Author: LHammonds
    ############# Update Log ###############
    # 2011-10-29 - LTH - Created schedule
    # 2011-11-22 - LTH - Added shell and path
    # 2012-01-30 - LTH - Added time adjustment
    ########################################
    
    SHELL=/bin/sh
    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
    
    # m h dom mon dow command
    
    #
    # Get date/time from network time server every hour
    #
    0 * * * * /etc/network/if-up.d/ntpdate
    #
    # Import Active Directory users (run every minute)
    0-59 * * * * /var/scripts/prod/import-ad.sh > /dev/null 2>&1
    #
    # Update the Global Address List for iPhone usage.
    #
    0 23 * * * /var/scripts/prod/gal-sync.sh > /dev/null 2>&1
    #
    # Partial backup of Zimbra to a local folder without taking it offline.
    #
    0 8-18 * * * /var/scripts/prod/rsync-online.sh > /dev/null 2>&1
    #
    # Backup Zimbra to a local folder, archive and store offsite.
    #
    0 23 * * * /var/scripts/prod/rsync-offline.sh > /dev/null 2>&1
    #
    # Backup individual mailboxes to a local folder, archive and store offsite.
    #
    0 6,12,18 * * * /var/scripts/prod/mailbox-backup.sh > /dev/null 2>&1
    Once you have created the file, make sure appropriate permissions are set by typing the following:
    Code:
    chown root:root /var/scripts/prod/crontab.root
    chmod 0600 /var/scripts/prod/crontab.root
    To enable the root schedule using this file, type the following:

    Code:
    crontab -u root /var/scripts/prod/crontab.root
    To disable the root schedule, type the following:
    Code:
    touch /tmp/deleteme
    crontab -u root /tmp/deleteme
    rm /tmp/deleteme
    If you need to modify the schedule, make a backup copy 1st. For example:

    Code:
    cp /var/scripts/prod/crontab.root /var/scripts/prod/2011-11-28-crontab.root
    vi /var/scripts/prod/crontab.root (make your changes)
    crontab -u root /var/scripts/prod/crontab.root
    LHammonds
    Last edited by LHammonds; 01-30-2012 at 02:59 PM.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  10. #20
    LHammonds's Avatar
    LHammonds is offline Special Member
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    3

    Default

    I had an initial, internal review of the system. My boss wants a mailbox size notification setup to remind people that their mailbox is approaching or reached the mailbox size limitation. However, he does not want any restrictions enforced...meaning that these are email notifications only and will not actually hinder use of the system. Idle threats basically. He also wants certain people (admin) excluded from these notifications and also wants to allow different threshholds for some people...such as everyone having a 100 MB limit, managers having a 200 MB limit and himself with a 300 MB limit.

    Since this is to be idle threats only, it seems setting quotas would be overkill (or at least added complexity) and may cause usage problems (if the quotas were enforced). I simply don't know at this time but my gut feeling is that I need to create a script that will check each mailbox size and then evaluate person based on an exception rule set.

    I would imagine the "exception" file would contain a list of accounts and rules to be applied.

    Example exception file:
    Code:
    ;AccountName, MaxSizeInMB, Exempt?
    ceoDude, 0, Y
    cioDude, 0, Y
    hrDude, 0, Y
    myboss, 300, N
    bossfriend, 200, N
    To get a list of users and how much space they are using, type "/opt/zimbra/bin/zmprov gqu mail"

    Useful links:
    - Qutoa Warning Script

    I'll update this post with whatever solution I end up implementing.
    Last edited by LHammonds; 11-23-2011 at 07:53 AM.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

Page 2 of 4 FirstFirst 1234 LastLast

Thread Information

Users Browsing this Thread

There are currently 3 users browsing this thread. (0 members and 3 guests)

Similar Threads

  1. zmperditionctl start asking for password
    By k7sle in forum Administrators
    Replies: 32
    Last Post: 02-20-2008, 11:13 AM
  2. Replies: 31
    Last Post: 12-15-2007, 09:05 PM
  3. Post instsallation problems
    By Assaf in forum Installation
    Replies: 14
    Last Post: 01-29-2007, 11:38 AM
  4. 3.1 on FC4 problems
    By cohnhead in forum Installation
    Replies: 8
    Last Post: 05-26-2006, 11:16 AM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •