Page 30 of 66 FirstFirst ... 20282930313240 ... LastLast
Results 291 to 300 of 659

Thread: [SOLVED] Yet Another Backup Script for Community Version

  1. #291
    liverpoolfcfan's Avatar
    liverpoolfcfan is offline Outstanding Member
    Join Date
    Oct 2009
    Location
    Dublin, IRELAND
    Posts
    710
    Rep Power
    6

    Default

    Quote Originally Posted by LaFong View Post
    You can restore a user by:

    Detailed instructions removed to save space - click on the little graphic to go back to original post for details - Post #235 on page 24 of this thread
    Never had to do it yet either (touch wood) - Very keen to hear how it goes. Please reply and let us know.

    Good luck :-)

  2. #292
    LaFong is offline Advanced Member
    Join Date
    Nov 2008
    Location
    Denver, CO
    Posts
    221
    Rep Power
    6

    Default

    Note that if she just deleted mail from a single mailbox, you can just restore the mails in that mailbox, adding "X-Zimbra-Received" and using addMessage as described, without having to completely recreate her as a user. The method I described there will lose any Calendar, Task, etc info. It's also possible to get double messages after importing messages, if they still exist in the mailbox.

    Someone should figure out the full procedure for restoring Calendars, Tasks, Address Books, etc, for a user.

  3. #293
    borngunners is offline Advanced Member
    Join Date
    Jun 2009
    Posts
    195
    Rep Power
    6

    Default

    Thanks LaFong for your response, but I need the step by step procedure. For now, I am just concerned about the mails in her inbox and not worried about calendars address book or any of the sort. If you could demonstarte the procedure to get the messages from the backup-file zimbrabackup_041320010.diff.1.dar...etc. I will really appreciate it.

    thanks

  4. #294
    pilcheck is offline Starter Member
    Join Date
    Apr 2010
    Posts
    2
    Rep Power
    5

    Default

    Since this seems to be the spot for this _stellar_ script, I thought I would post a few comments /quirks

    First, to borngunners, couldn't you extract only her inbox directory with dar?
    Something along the lines of dar -x /Backup/foo.dar -g /opt/zimbra/store/rest/of/path?


    In my testing of this script I came across this, but the backup still completed.
    Code:
    diff Backup started at: Fri Apr 16 09:55:31 EDT 2010
    ./zmbak_v.0.8.sh: line 592: [: /Backup/zimbra_dars/15_ZimbraBackup_20100415_FULL: binary operator expected
    ./zmbak_v.0.8.sh: line 600: [: /Backup/zimbra_dars/15_ZimbraBackup_20100415_FULL: binary operator expected
    And I also came across this:
    Code:
    Aborting program. User refused to continue while asking: /Backup/zimbra_dars/15_ZimbraBackup_20100415_FULL
    /Backup/zimbra_dars/15_ZimbraBackup_20100416_FULL.1.dar is required for further operation, please provide the file.
    Dar had a problem!
    [root@mail scripts]# ls -hal /Backup/zimbra_dars/
    total 1.3G
    drwxr-xr-x 2 root root 4.0K Apr 16 09:34 .
    drwxr-xr-x 4 root root 4.0K Apr 15 14:19 ..
    -rw-r--r-- 1 root root 585M Apr 15 14:59 15_ZimbraBackup_20100415_FULL.1.dar
    -rw-r--r-- 1 root root   70 Apr 15 14:59 15_ZimbraBackup_20100415_FULL.1.dar.md5
    -rw-r--r-- 1 root root 459K Apr 15 14:59 15_ZimbraBackup_20100415_FULL.txt.gz
    -rw-r--r-- 1 root root 661M Apr 16 09:34 15_ZimbraBackup_20100416_FULL.1.dar
    -rw-r--r-- 1 root root   70 Apr 16 09:34 15_ZimbraBackup_20100416_FULL.1.dar.md5
    -rw-r--r-- 1 root root 476K Apr 16 09:34 15_ZimbraBackup_20100416_FULL.txt.gz
    That was resolved by removing the FULL from the same day. Can we maintain a FULL and DIFF of the same day while keeping the files in the same directory?

    Thanks for the hard work folks!

  5. #295
    liverpoolfcfan's Avatar
    liverpoolfcfan is offline Outstanding Member
    Join Date
    Oct 2009
    Location
    Dublin, IRELAND
    Posts
    710
    Rep Power
    6

    Default

    You should be able to do a DIFF and FULL from the same day. You just cannot have two different FULL backups in the base folder.

    The reason being that the backup script needs to know which FULL backup is the starting point for the generation of the DIFFerential backup. It is recording all changes since the (expecting only one possible) FULL backup was made.

    If you wish to do daily FULL backups, and one or multiple DIFFs during the day, you would need to modify the script so it no longer works off WEEK numbers, but instead works off DAY numbers.

    Or otherwise, just remove the week number checking at the beginning of the script, and always move files to the media folder whenever a FULL backup is run, so the main backup_dars folder is cleared out daily.
    Last edited by liverpoolfcfan; 04-16-2010 at 09:02 AM. Reason: corrected spelling

  6. #296
    LaFong is offline Advanced Member
    Join Date
    Nov 2008
    Location
    Denver, CO
    Posts
    221
    Rep Power
    6

    Default Restoring a single folder

    Quote Originally Posted by borngunners View Post
    Thanks LaFong for your response, but I need the step by step procedure. For now, I am just concerned about the mails in her inbox and not worried about calendars address book or any of the sort. If you could demonstarte the procedure to get the messages from the backup-file zimbrabackup_041320010.diff.1.dar...etc. I will really appreciate it.
    You don't have an rsync directory, or it's already been overwritten (removing the accidentally deleted emails)?

    Ugh. The thing got more complicated the more I thought about it. I hope I haven't forgotten anything else.

    Restoring a single folder

    I have not thoroughly tested this procedure, use at your own risk. If restoring to your existing Zimbra server, make sure you have a backup.

    • This procedure only works on the Inbox or IMAP folders.
    • Most, if not all, the steps can/should be run as the zimbra user, with the full zimbra environment, accomplished this way:
      Code:
      [root@mail bin]# su - zimbra
      [zimbra@mail bin]# source ~/bin/zmshutil ; zmsetvars
    • If you run a command or script which writes output to a folder (other than /tmp), the zimbra user must have write permissions.


    You can restore a folder's messages by:

    1) Create a temporary restore folder in the user's IMAP tree, e.g. "Restore_Temp". If the user has messages in their existing folder which are also in the backup, those messages would get doubled, so it's easiest if you restore the old folder messages to a temporary folder.

    2) Find the user's mailboxId:
    Code:
    [zimbra@mail bin]# zmprov getMailboxInfo user@domain.com
    
    mailboxId: 8
    quotaUsed: 12350
    Note all the paths below which use the mailboxId, "8", in the path. Change that to the desired user's mailboxId.

    3) Find out the folder ID for the folder you want to restore.
    Code:
    zmmailbox -z -m user@somewhere.com gaf
    The Id for the Inbox is typically "2".

    Code:
    [zimbra@mail ~]$ zmmailbox -z -m user@somewhere.com gaf
            Id  View      Unread   Msg Count  Path
    ----------  ----  ----------  ----------  ----------
             1  conv           0           0  /
           282  mess           0           0  /Big Test
            16  docu           0           0  /Briefcase
            10  appo           0           1  /Calendar
            14  mess           0           0  /Chats
             7  cont           0           0  /Contacts
           264  mess           1           1  /crap
             6  mess           0           0  /Drafts
            13  cont           0           0  /Emailed Contacts
           266  mess           1           1  /forward
             2  mess           3          17  /Inbox
             4  mess           0           0  /Junk
           257  mess           0           0  /Junk E-mail
           268  mess           0           0  /mail
           269  mess           0           0  /mail/Drafts
           270  mess           0           2  /mail/Sent
           273  mess           0           0  /mail/Trash
           320  conv           0           0  /MyFolderTest
            12  wiki           0           0  /Notebook
             5  mess           0           4  /Sent
            15  task           0           0  /Tasks
           262  mess           0           1  /Test
             3  conv           0           1  /Trash
    4a) If the rsync directory exists and has all required emails, copy the user's backup store to a new location, for the next step.
    Code:
    [zimbra@mail ~]$ cp -ar /backup/sync/store/0/8/msg /home
    4b) If the rsync directory does not exist, or is missing the required emails, restore the dar files to a new directory:
    i) Create temp restore directory:
    Code:
    [zimbra@mail bin]# mkdir /home/dar_restore
    ii) Change directory to temp restore location!
    Code:
    [zimbra@mail bin]# cd /home/dar_restore
    iii) Restore user's store backup and db to tempdir.
    Store:
    • Make sure you have enough room on the partition, as this grabs all their mail not just the folder you are restoring.
    • You need just the full and desired (usually latest) diff.
    • These commands restore to the current directory. To restore elsewhere, use -R option.

    Code:
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g store/0/8/msg
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g store/0/8/msg
    Databases:
    • Necessary only if you do not have mysqldump's of db's.
    • If you don't have sql dumps of users' dbs, extract the backups of the db files.
    • You will need the user's db directory (e.g. db/data/mboxgroup8), the mysql db directory, and any files in db/data whose names begin with "ib".
    • You need just the full and desired (usually latest) diff.
    • These commands restore to the current directory. To restore elsewhere, use -R option.

    Code:
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/mboxgroup8
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/mboxgroup8
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/mysql
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/mysql
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/ibdata1
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/ibdata1
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/ib_logfile0
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/ib_logfile0
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/ib_logfile1
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/ib_logfile1
    Use the correct mboxgroup for the user, in this case mboxgroup8.

    5) Recreate old db, in order to grab message_ids. There are 2 ways, in order of preference:
    a) Schedule regular mysql dumps of user mailboxes, perhaps incorporated into the backup script itself. Then restore the dump to a temp db. (MySQL, including the one in Zimbra, contains an empty "test" db, by default. We will use that.) OR
    b) Use a different MySQL server to query the user's old db.

    a) If you have a sql dump, you can restore to the test db:
    Set the zimbra environment...
    Code:
    su - zimbra
    source ~/bin/zmshutil ; zmsetvars
    If you have the processed databases, using the zmDBbac.sh script, and don't have the raw mysql dumps, convert the dump back to raw If you have the raw DB already, skip this step.:
    Code:
    [zimbra@mail tmp]$ /usr/local/sbin/zmDBbac.sh makeraw /backup/sync/db_dumps/mailboxes/8.sql 8-raw.sql
    then import the user's db backup into the 'test' db:
    Code:
    [zimbra@mail tmp]$ mysql test < 8-raw.sql
    You should not have to restart Zimbra if you do sql dumps, so this method is easier all around, which argues for including it in the backup script, as I have done here: Adding per user backup and restore. (Dumping all the dbs is discussed here.)

    OR

    b)If you are using a different MySQL server for this step, stop the mysqld service, mv /var/lib/mysql /var/lib/mysql.bak, create a new /var/lib/mysql directory, copy from backup/db/data the old db directory, the mysql directory, all ib_logfile and ibdata files, chown -R mysql:mysql /var/lib/mysql, and restart the mysql service. Copy the user's backup store to a suitable location for the next step. You may need to add this line to the [mysqld] section of my.cnf:
    Code:
    innodb_log_file_size      = 104857600
    6) Run the following script that grabs the index_ids for the desired folder, does a find for the message files based on those index_ids, then copies all of the messages into a RESTORE directory in the current directory. "It works for me", but I only tried it on a small folder.

    Run in a directory you have write privileges to, on a partition with enough space. Example (for user "8"):
    If you are restoring the user's db into the 'test' db on your live Zimbra server...
    Code:
    /path/to/script/recreate_Folder.sh test 2 /home/msg
    If you are using the 5b method instead, and restoring the user to another Zimbra server:
    Code:
    /path/to/script/recreate_Folder.sh 8 2 /home/msg
    In either case, "8" being the user's mailboxId from Step 2 and "2" being the folder Id from Step 3.

    Code:
    #!/bin/bash
    # USAGE:    -h or --help for help & usage.
    #           recreate_Folder.sh mailbox folderId /path/to/folder/containing/msg
    #           e.g
    #           recreate_Folder.sh test 2 /backup/store/0/8/msg
    #           or, if not using the 'test' db, and restoring to a different server
    #           recreate_Folder.sh 8 2 /backup/store/0/8/msg
    mailboxId="$1"
    folderId="$2"
    BackupStore="$3"
    RESTORE=/tmp/RESTORE     # path to location of restored message files. Set to a location with enough room.
    
    function checkUser {
        if [ $USER != "zimbra" ] ; then
           echo "Run script as the zimbra user!"
           exit 1
        fi
        checkVariables
    }
    
    function checkVariables {
        if [ -z "$mailboxId" ] || [ -z "$folderId" ] || [ -z "BackupStore" ] ; then
           show_help
           exit 1
        else
           checkDirectory
        fi
    }
    
    function checkDirectory {
        if ! [ -d "$BackupStore" 2>/dev/null ] ; then
           echo
           echo "Message directory does not exist!"
           show_help
           exit 0
        elif [ -d ./RESTORE 2>/dev/null ] ; then
           echo
           echo "RESTORE folder already exists!"
           show_help
           exit 0
        else
           mboxgroupDB
        fi
    }
    
    function mboxgroupDB {
    if (("$mailboxId" > 100)) ; then
        if ((`echo "$mailboxId" | grep -o '..$' | sed 's/0*//'` > 9)) ; then
            mboxgroup=`echo "$mailboxId" | grep -o '..$'`
        else
            mboxgroup=`echo "$mailboxId" | grep -o '.$'`
        fi
    else
        mboxgroup="$mailboxId"
    fi
    addMessages
    }
    
    function addMessages {
    source ~/bin/zmshutil ; zmsetvars
    if [ -f /tmp/Folder_Msgs ] ; then
      rm /tmp/Folder_Msgs
    fi
    mkdir -p "$RESTORE"/msg/0
    cd "$BackupStore" 
    if [ "$mailboxId" = "test" ] ; then
       mysql -N -e "SELECT index_id FROM mail_item WHERE folder_id="$folderId" AND mailbox_id="$mailboxId test | grep -v + > /tmp/Folder_Ids
    else
       mysql -N -e "SELECT index_id FROM mail_item WHERE folder_id="$folderId" AND mailbox_id="$mailboxId mboxgroup$mboxgroup | grep -v + > /tmp/Folder_Ids
    fi
    while read index_id ; do
      find "$BackupStore" -print | grep "/$index_id-" >> /tmp/Folder_Msgs
    done < /tmp/Folder_Ids
    while read message ; do
      cp -p "$message" "$RESTORE"/msg/0
    done < /tmp/Folder_Msgs
    }
    
    function show_help {
        echo
        echo "USAGE:    recreate_Folder.sh test folderId /path/to/folder/containing/msg"  OR
        echo "USAGE:    recreate_Folder.sh mailboxId folderId /path/to/folder/containing/msg"
        echo "          e.g"
        echo "          recreate_Folder.sh test 2 /home/msg"
        echo "          recreate_Folder.sh 8 2 /home/msg"
        echo "          -h or --help for help & usage."
        echo
        echo
    }
    
    case $1 in
    -H | -h | --help)
            show_help
    ;;
    *)
            checkUser
    ;;
    esac
    exit 0
    7) Create a script to copy the "Date" header in every message into a new "X-Zimbra-Received" header before you do the import. The messages will be imported as if they'd been received when the message was sent rather than when they were imported.

    Code:
    [zimbra@mail ~]$ /path/to/add_received.sh /path/to/restored/msg
    The path should be to the msg directory within the RESTORE path from the previous step.

    Code:
    #!/bin/bash
    # USAGE:    -h or --help for help & usage.
    #           add_received.sh path/to/folder/containing/msg
    #           e.g
    #           add_received.sh /tmp/RESTORE/msg
    
    BackupStore="$1"
    
    function checkDirectory {
        if [ -d "$BackupStore" 2>/dev/null ] ; then
           addReceived
        else
           echo
           echo "Directory does not exist!"
           show_help
           exit 0
        fi
    }
    
    function addReceived {
        for d in `ls -A -1 "$BackupStore"` 
          do
            HASMSG=`ls -A -1 "$BackupStore/$d"/*.msg | head -1 2>/dev/null`
            if [ -z $HASMSG ] ; then
               echo
               echo "Specified directory has no .msg files!"
               show_help
            else
                for i in `ls -A -1 "$BackupStore/$d"/*.msg`
                   do
                       if ! grep "X-Zimbra-Received" $i 1>/dev/null ; then  
                            sed -i /^Date/p $i
                            sed -i '0,/^Date/s/^Date/X-Zimbra-Received/' $i
                       fi
                   done
            fi
          done
    }
    
    function show_help {
        echo
        echo "USAGE:    add_received.sh /path/to/folder/containing/msg"
        echo "          e.g"
        echo "          add_received.sh /tmp/RESTORE/msg"
        echo "          -h or --help for help & usage."
        echo
        echo
    }
    
    case $1 in
    -H | -h | --help)
            show_help
    ;;
    *)
            checkDirectory
    ;;
    esac
    exit 0
    8) Restore their folder messages to "Restore_Temp". As the Zimbra user:
    Code:
    [zimbra@mail ~]$ zmprov
    prov> sm user@somewhere.com
    mailbox: user@somewhere.com, size: 26.29 KB, messages: 12, unread: 4
    mbox user@somewhere.com>addMessage Restore_Temp /tmp/RESTORE/msg/0
    The path should be msg/0 within the RESTORE path of previous steps. (This msg/0 subfolder was added purely to allow header processing of multiple message folders.)

    9) The user then can either just rename the Restore_Temp folder in their client, or compare their messages in the Restore_Temp folder in their email client vs their existing folder, remove doubles, and move the remaining messages from the Restore_Temp folder back to the desired folder, and delete the Restore_Temp folder.

    10) If you used the 'test' db method, drop and recreate the empty test db, for the next time:
    As Zimbra user:
    Code:
    source ~/bin/zmshutil ; zmsetvars
    ~/mysql/bin/mysqladmin -S $mysql_socket drop test
    ~/mysql/bin/mysqladmin -S $mysql_socket create test
    Last edited by LaFong; 11-29-2011 at 06:31 PM.

  7. #297
    blessendor is offline Active Member
    Join Date
    Dec 2009
    Posts
    46
    Rep Power
    5

    Default Another path to data folders

    Thanks authors for their script!
    I will be test it today.

    But for now I read script row by row and I think you can do some changes in the code for users like me: I don't have a real some data folders in the /opt/zimbra - just only sym links.

    So, can you add in your script next VARs (if exist settings or "" for ignore):

    ZM_STORE_DIR="" # Type your real path to message "store" folder, default - blank, that mean "/opt/zimbra/store"
    ZM_DATA_DIR="" # Same logic as above
    ZM_DB_DIR="" # Same logic as above

    ZM_BACKUP_DIR="" # Means standard /opt/zimbra/backup dir if it will be used by your script or can be ignored, because /opt/zimbra/backup is empty (for me).

    For my server this settings are:

    ZM_STORE_DIR="/mnt/RAID10/store"
    ZM_DATA_DIR="/mnt/RAID10/data"
    ZM_DB_DIR="/mnt/RAID10/db"
    ZM_BACKUP_DIR="/mnt/RAID10/backup"

    Sorry for my English. I hope you understood me.

  8. #298
    LaFong is offline Advanced Member
    Join Date
    Nov 2008
    Location
    Denver, CO
    Posts
    221
    Rep Power
    6

    Default

    Quote Originally Posted by blessendor View Post
    I don't have a real some data folders in the /opt/zimbra - just only sym links.
    So, can you add in your script next VARs (if exist settings or "" for ignore):

    ZM_STORE_DIR="" # Type your real path to message "store" folder, default - blank, that mean "/opt/zimbra/store"
    ZM_DATA_DIR="" # Same logic as above
    ZM_DB_DIR="" # Same logic as above
    I'm not sure it's worth updating the script for all these separate volumes, it's kinda old school. You could make an argument for the store directory. The data and db directories are not that large.

    ZM_BACKUP_DIR="" # Means standard /opt/zimbra/backup dir if it will be used by your script or can be ignored, because /opt/zimbra/backup is empty (for me).
    This directory is used by the Network Edition, not FOSS. (Mine actually does have old ldap config backups, total size 1MB). It would not be worth symlinking or making a separate volume, unless you plan on upgrading to the NE version.

    For my server this settings are:

    ZM_STORE_DIR="/mnt/RAID10/store"
    ZM_DATA_DIR="/mnt/RAID10/data"
    ZM_DB_DIR="/mnt/RAID10/db"
    ZM_BACKUP_DIR="/mnt/RAID10/backup"
    You could just modify the existing rsync commands like:
    Code:
    nice -19 $RSYNC_BIN $RSYNC_OPTS $ZM_HOME /mnt/RAID10 $SYNC_DIR
    This will create a RAID10 directory within your SYNC_DIR containing your store, db, and data.

    If there are volumes on RAID10 you want to exclude, create the /opt/zimbra/excludes file containing just their directory names, i.e. instead of "/mnt/RAID10/unwanted", just put "unwanted". One line per excluded volume. Then modify the RSYNC_OPTS line:
    Code:
    RSYNC_OPTS="-aHK --delete --exclude=*.pid --delete-excluded --exclude-from=/opt/zimbra/excludes" # leave these unless you are sure you need something else
    Last edited by LaFong; 04-26-2010 at 08:55 PM.

  9. #299
    borngunners is offline Advanced Member
    Join Date
    Jun 2009
    Posts
    195
    Rep Power
    6

    Default

    I am trying to follow the procedure on a backup server. I am confuse with number 4, so I skipped it because I am doing it on a new backup server that does not have any email. I just use it to store my backup email even though zimbra is installed on it. When I try to do number 5 as a zimbra user, this is what I got:
    Code:
    zimbra@mail:/var/tmp/restore$ recreate_inbox.sh test@account.mail.com /opt/zimbra/store/0/23/msg
    bash: recreate_inbox.sh: command not found
    zimbra@mail:/var/tmp/restore$ exit
    I am not sure if I am running the script the right way.

    thanks
    Quote Originally Posted by LaFong View Post
    You don't have an rsync directory, or it's already been overwritten (removing the accidentally deleted emails)?

    Ugh. The thing got more complicated the more I thought about it. I hope I haven't forgotten anything else.

    I have not thoroughly tested this procedure, use at your own risk.

    You can restore her Inbox messages by:

    1) Optional: Delete or move messages that would get doubled. If the user has messages in their Inbox which are also in the Inbox backup, those messages will get doubled when you import the Inbox backup. It is probably easiest if the user just moves all of their Inbox messages into a backup folder within Zimbra, or other client.

    2) Find the user's mailboxId:
    Code:
    [zimbra@mail bin]# zmprov getMailboxInfo user@domain.com
    
    mailboxId: 8
    quotaUsed: 12350
    Note all the paths below which use the mailboxId, "8", in the path. Change that to the desired user's mailboxId.

    3a) If the rsync directory exists and has all required emails, copy the user's backup store to a new location, for the next step.
    Code:
    [zimbra@mail ~]$ cp -pr /backup/sync/store/0/8/msg /home
    3b) If the rsync directory does not exist, or is missing the required emails, restore the dar files to a new directory:
    i) Create temp restore directory:
    Code:
    [zimbra@mail bin]# mkdir /home/restore
    ii) Change directory to temp restore location!
    Code:
    [zimbra@mail bin]# cd /home/restore
    iii) Restore user's folder backup and db to tempdir:
    Code:
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g store/0/8/msg
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04132010_DIFF -w -g store/0/8/msg
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g store/0/8/msg
    If you don't have sql dumps of users' dbs, extract the backups:
    Code:
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/mboxgroup8
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04132010_DIFF -w -g db/data/mboxgroup8
    [zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/mboxgroup8
    Otherwise, extract or retrieve the dump for that user.
    Make sure you have enough room on the partition, as this grabs all their mail not just the Inbox.

    4) Recreate old db, in order to grab message_ids. There are 3 ways, in order of preference:
    a) Schedule regular mysql dumps of user mailboxes, perhaps incorporated into the backup script itself. Then restore the dump to a temp db.
    b) Use a different MySQL server to query the user's old db.
    c) Use the Zimbra server.

    a) If you have a sql dump, you can restore to the test db:
    Code:
    mysql test <mboxgroup8_04142010.sql
    You should not have to restart Zimbra if you do sql dumps, so this method is easier all around, which argues for including it in the backup script. (Dumping all the dbs is discussed here.) Change the script below to use 'test' instead of 'mboxgroup$mailboxId' if you use the test db method.

    OR

    b)If you are using a different MySQL server for this step, just create an empty 'mboxgroup8' db (in this case), stop the mysqld service, copy the old db directory over it, and restart the service. You will need to adjust the recreate_Inbox.sh script below to not grab the mail files, but just generate the Inbox message_ids. This can be done by commenting out the find and cp lines. Then, on the Zimbra server, copy the Inbox_Ids file to /tmp, change the script by commenting out the mysql line, and uncommenting the find and cp lines.

    OR

    c)Backup existing and restore the old user's mailbox db, retrieve message_ids, restore existing db. Any messages that arrive for the user during this period will be lost when the existing db is restored. Thus, this is the least preferred method.
    Code:
    /opt/zimbra/bin/zmcontrol stop
    cp -pr /opt/zimbra/db/data/mboxgroup8 /home
    cp -pr /home/restore/db/data/mboxgroup8/* /opt/zimbra/db/data/mboxgroup8
    /opt/zimbra/bin/zmcontrol start
    Run the script in the next step, then restore the existing db:
    Code:
    /opt/zimbra/bin/zmcontrol stop
    rm -fr /opt/zimbra/db/data/mboxgroup8
    mv /home/mboxgroup8 /opt/zimbra/db/data
    /opt/zimbra/bin/zmcontrol start
    5) Run the following script that grabs the index_ids for the Inbox, does a find for the message files based on those index_ids, then copies all of the messages into an INBOX directory in the current directory. "It works for me", but I only tried it on a small Inbox.

    Run as the Zimbra user, in a directory you have write privileges to, on a partition with enough space.
    Code:
    recreate_Inbox.sh joe@company.com /home/restore/store/0/8/msg
    Code:
    #!/bin/sh
    # USAGE:    -h or --help for help & usage.
    #           recreate_Inbox.sh user@domain.com path/to/folder/containing/msg/directory
    #           e.g
    #           recreate_Inbox.sh joe@company.com /home/restore/store/0/8/msg
    
    User="$1"
    BackupStore="$2"
    PWD=`pwd`
    INBOX="$PWD"/INBOX
    
    function checkUser {
        if [ -n "$User" 2>/dev/null ] ; then
           checkDirectory
        else
           echo
           echo "No User Specified!"
           show_help
           exit 0
        fi
    }
    
    function checkDirectory {
        if ! [ -d "$BackupStore" 2>/dev/null ] ; then
           echo
           echo "Message directory does not exist!"
           show_help
           exit 0
        elif [ -d ./INBOX 2>/dev/null ] ; then
           echo
           echo "INBOX directory already exists!"
           show_help
           exit 0
        else
           addMessages
        fi
    }
    
    function addMessages {
    if [ -f /tmp/Inbox_Msgs ] ; then
      rm /tmp/Inbox_Msgs
    fi
    mkdir INBOX
    cd "$BackupStore"
    mailboxId=`zmprov getMailboxInfo "$User" | grep mailboxId | awk '{print $2}'`
    mysql -N -e "SELECT index_id FROM mail_item WHERE folder_id=2 mboxgroup$mailboxId" | grep -v + > /tmp/Inbox_Ids
    while read index_id ; do
      find "$BackupStore" -print | grep "/$index_id-" >> /tmp/Inbox_Msgs
    done < /tmp/Inbox_Ids
    while read message ; do
      cp -p "$message" "$INBOX"
    done < /tmp/Inbox_Msgs
    }
    
    function show_help {
        echo
        echo "USAGE:    recreate_Inbox.sh user@domain.com path/to/folder/containing/msg/directory"
        echo "          e.g"
        echo "          recreate_Inbox.sh joe@company.com /home/restore/store/0/8/msg"
        echo "          -h or --help for help & usage."
        echo
        echo
    }
    
    case $1 in
    -H | -h | --help)
            show_help
    ;;
    *)
            checkUser
    ;;
    esac
    exit 0
    6) Create a script to copy the "Date" header in every message into a new "X-Zimbra-Received" header before you do the import. The messages will be imported as if they'd been received when the message was sent rather than when they were imported. Here is a rough script which I have not thoroughly debugged. Use at your own risk. Do NOT use on the live Zimbra store, or on your pristine backup. Check the results before trying to import messages.

    Code:
    [zimbra@mail ~]$ add_received.sh /tmp/INBOX
    (Change target path to suit.)

    Code:
    #!/bin/sh
    # USAGE:    -h or --help for help & usage.
    #           add_received.sh path/to/folder/containing/msg/directory
    #           e.g
    #           add_received.sh /home/msg
    
    BackupStore="$1"
    
    function checkDirectory {
        if [ -d "$BackupStore" 2>/dev/null ] ; then
           addReceived
        else
           echo
           echo "Directory does not exist!"
           show_help
           exit 0
        fi
    }
    
    function addReceived {
        for d in `ls -A -1 "$BackupStore"` 
          do
            HASMSG=`ls -A -1 "$BackupStore/$d"/*.msg | head -1 2>/dev/null`
            if [ -z $HASMSG ] ; then
               echo
               echo "Specified directory has no .msg files!"
               show_help
            else
                for i in `ls -A -1 "$BackupStore/$d"/*.msg`
                   do
                       if ! grep "X-Zimbra-Received" $i 1>/dev/null ; then  
                            sed -i /^Date/p $i
                            sed -i '0,/^Date/s/^Date/X-Zimbra-Received/' $i
                       fi
                   done
            fi
          done
    }
    
    function show_help {
        echo
        echo "USAGE:    add_received.sh path/to/folder/containing/msg/files"
        echo "          e.g"
        echo "          add_received.sh /home/msg_backup"
        echo "          -h or --help for help & usage."
        echo
        echo
    }
    
    case $1 in
    -H | -h | --help)
            show_help
    ;;
    *)
            checkDirectory
    ;;
    esac
    exit 0
    7) Restore their Inbox. As the Zimbra user:
    Code:
    [zimbra@mail ~]$ zmprov
    prov> sm joe@company.com
    mailbox: joe@company.com, size: 26.29 KB, messages: 12, unread: 4
    mbox joe@company.com>addMessage INBOX /tmp/INBOX

    8) The user then compares her Inbox messages in the backup folder vs her Inbox, removes doubles, and moves the remaining messages from the temp folder back to the Inbox, and deletes the temp folder.

    9) If you used the 'test' db method, drop and recreate the empty test db, for the next time:
    Code:
    source ~/bin/zmshutil ; zmsetvars
    ~/mysql/bin/mysqladmin -S $mysql_socket -u root --password=$mysql_root_password drop test
    ~/mysql/bin/mysqladmin -S $mysql_socket -u root --password=$mysql_root_password create test

    You'll have to figure out any path differences, etc, for your situation.

    This procedure could be adapted to other folders. Their ids may be determined as follows:
    Code:
    mysql mboxgroup$mailboxId -N -e "SELECT id, name FROM mail_item WHERE type=1" | grep -v +
    e.g.
    Code:
    mysql mboxgroup8 -N -e "SELECT id, name FROM mail_item WHERE type=1" | grep -v +
    Somehow, Zimbra mixes "id" and "folder_id". You use the id values for folder_id lookup of index_ids. I have no idea why.

  10. #300
    LaFong is offline Advanced Member
    Join Date
    Nov 2008
    Location
    Denver, CO
    Posts
    221
    Rep Power
    6

    Default

    Quote Originally Posted by borngunners View Post
    I am trying to follow the procedure on a backup server. I am confuse with number 4, so I skipped it because I am doing it on a new backup server that does not have any email. I just use it to store my backup email even though zimbra is installed on it.

    When I try to do number 5 as a zimbra user, this is what I got:
    Code:
    zimbra@mail:/var/tmp/restore$ recreate_inbox.sh test@account.mail.com /opt/zimbra/store/0/23/msg
    bash: recreate_inbox.sh: command not found
    zimbra@mail:/var/tmp/restore$ exit
    You don't have the path. So run it with it included:
    Code:
    /path/to/script/recreate_inbox.sh test@account.mail.com /opt/zimbra/store/0/23/msg

Page 30 of 66 FirstFirst ... 20282930313240 ... LastLast

Thread Information

Users Browsing this Thread

There are currently 2 users browsing this thread. (0 members and 2 guests)

Similar Threads

  1. Zimbra Backup Script ... Small Problem
    By frankb in forum Administrators
    Replies: 12
    Last Post: 07-14-2008, 08:40 AM
  2. Debian - Upgrade from 4.5.6
    By magikman in forum Installation
    Replies: 1
    Last Post: 01-19-2008, 12:59 PM
  3. Backup script issues
    By SSS in forum Administrators
    Replies: 18
    Last Post: 12-03-2007, 09:56 AM
  4. 4.0.4 to 4.5.6 upgrade failed in network edition
    By chenthil in forum Administrators
    Replies: 1
    Last Post: 08-27-2007, 09:36 AM
  5. [SOLVED] Simple backup question...
    By dameron in forum Administrators
    Replies: 3
    Last Post: 08-25-2007, 09:36 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •