[SOLVED] Yet Another Backup Script for Community Version

[liverpoolfcfan]

You can restore a user by:

Detailed instructions removed to save space - click on the little graphic to go back to original post for details - Post #235 on page 24 of this thread

Never had to do it yet either (touch wood) - Very keen to hear how it goes. Please reply and let us know.

Good luck :-)

[LaFong]

Note that if she just deleted mail from a single mailbox, you can just restore the mails in that mailbox, adding "X-Zimbra-Received" and using addMessage as described, without having to completely recreate her as a user. The method I described there will lose any Calendar, Task, etc info. It's also possible to get double messages after importing messages, if they still exist in the mailbox.

Someone should figure out the full procedure for restoring Calendars, Tasks, Address Books, etc, for a user.

[borngunners]

Thanks LaFong for your response, but I need the step by step procedure. For now, I am just concerned about the mails in her inbox and not worried about calendars address book or any of the sort. If you could demonstarte the procedure to get the messages from the backup-file zimbrabackup_041320010.diff.1.dar...etc. I will really appreciate it.

thanks

[pilcheck]

Since this seems to be the spot for this _stellar_ script, I thought I would post a few comments /quirks

First, to borngunners, couldn't you extract only her inbox directory with dar?
Something along the lines of dar -x /Backup/foo.dar -g /opt/zimbra/store/rest/of/path?


In my testing of this script I came across this, but the backup still completed.

diff Backup started at: Fri Apr 16 09:55:31 EDT 2010
./zmbak_v.0.8.sh: line 592: [: /Backup/zimbra_dars/15_ZimbraBackup_20100415_FULL: binary operator expected
./zmbak_v.0.8.sh: line 600: [: /Backup/zimbra_dars/15_ZimbraBackup_20100415_FULL: binary operator expected

And I also came across this:

Aborting program. User refused to continue while asking: /Backup/zimbra_dars/15_ZimbraBackup_20100415_FULL
/Backup/zimbra_dars/15_ZimbraBackup_20100416_FULL.1.dar is required for further operation, please provide the file.
Dar had a problem!
[root@mail scripts]# ls -hal /Backup/zimbra_dars/
total 1.3G
drwxr-xr-x 2 root root 4.0K Apr 16 09:34 .
drwxr-xr-x 4 root root 4.0K Apr 15 14:19 ..
-rw-r--r-- 1 root root 585M Apr 15 14:59 15_ZimbraBackup_20100415_FULL.1.dar
-rw-r--r-- 1 root root   70 Apr 15 14:59 15_ZimbraBackup_20100415_FULL.1.dar.md5
-rw-r--r-- 1 root root 459K Apr 15 14:59 15_ZimbraBackup_20100415_FULL.txt.gz
-rw-r--r-- 1 root root 661M Apr 16 09:34 15_ZimbraBackup_20100416_FULL.1.dar
-rw-r--r-- 1 root root   70 Apr 16 09:34 15_ZimbraBackup_20100416_FULL.1.dar.md5
-rw-r--r-- 1 root root 476K Apr 16 09:34 15_ZimbraBackup_20100416_FULL.txt.gz

That was resolved by removing the FULL from the same day. Can we maintain a FULL and DIFF of the same day while keeping the files in the same directory?

Thanks for the hard work folks!

[liverpoolfcfan]

You should be able to do a DIFF and FULL from the same day. You just cannot have two different FULL backups in the base folder.

The reason being that the backup script needs to know which FULL backup is the starting point for the generation of the DIFFerential backup. It is recording all changes since the (expecting only one possible) FULL backup was made.

If you wish to do daily FULL backups, and one or multiple DIFFs during the day, you would need to modify the script so it no longer works off WEEK numbers, but instead works off DAY numbers.

Or otherwise, just remove the week number checking at the beginning of the script, and always move files to the media folder whenever a FULL backup is run, so the main backup_dars folder is cleared out daily.

[LaFong]

Thanks LaFong for your response, but I need the step by step procedure. For now, I am just concerned about the mails in her inbox and not worried about calendars address book or any of the sort. If you could demonstarte the procedure to get the messages from the backup-file zimbrabackup_041320010.diff.1.dar...etc. I will really appreciate it.

You don't have an rsync directory, or it's already been overwritten (removing the accidentally deleted emails)?

Ugh. The thing got more complicated the more I thought about it. I hope I haven't forgotten anything else.

Restoring a single folder
I have not thoroughly tested this procedure, use at your own risk. If restoring to your existing Zimbra server, make sure you have a backup.

• This procedure only works on the Inbox or IMAP folders.
• Most, if not all, the steps can/should be run as the zimbra user, with the full zimbra environment, accomplished this way:
  

  [root@mail bin]# su - zimbra
  [zimbra@mail bin]# source ~/bin/zmshutil ; zmsetvars

• If you run a command or script which writes output to a folder (other than /tmp), the zimbra user must have write permissions.

You can restore a folder's messages by:

1) Create a temporary restore folder in the user's IMAP tree, e.g. "Restore_Temp". If the user has messages in their existing folder which are also in the backup, those messages would get doubled, so it's easiest if you restore the old folder messages to a temporary folder.

2) Find the user's mailboxId:

[zimbra@mail bin]# zmprov getMailboxInfo user@domain.com

mailboxId: 8
quotaUsed: 12350

Note all the paths below which use the mailboxId, "8", in the path. Change that to the desired user's mailboxId.

3) Find out the folder ID for the folder you want to restore.

zmmailbox -z -m user@somewhere.com gaf

The Id for the Inbox is typically "2".

[zimbra@mail ~]$ zmmailbox -z -m user@somewhere.com gaf
        Id  View      Unread   Msg Count  Path
----------  ----  ----------  ----------  ----------
         1  conv           0           0  /
       282  mess           0           0  /Big Test
        16  docu           0           0  /Briefcase
        10  appo           0           1  /Calendar
        14  mess           0           0  /Chats
         7  cont           0           0  /Contacts
       264  mess           1           1  /crap
         6  mess           0           0  /Drafts
        13  cont           0           0  /Emailed Contacts
       266  mess           1           1  /forward
         2  mess           3          17  /Inbox
         4  mess           0           0  /Junk
       257  mess           0           0  /Junk E-mail
       268  mess           0           0  /mail
       269  mess           0           0  /mail/Drafts
       270  mess           0           2  /mail/Sent
       273  mess           0           0  /mail/Trash
       320  conv           0           0  /MyFolderTest
        12  wiki           0           0  /Notebook
         5  mess           0           4  /Sent
        15  task           0           0  /Tasks
       262  mess           0           1  /Test
         3  conv           0           1  /Trash

4a) If the rsync directory exists and has all required emails, copy the user's backup store to a new location, for the next step.

[zimbra@mail ~]$ cp -ar /backup/sync/store/0/8/msg /home

4b) If the rsync directory does not exist, or is missing the required emails, restore the dar files to a new directory:
i) Create temp restore directory:

[zimbra@mail bin]# mkdir /home/dar_restore

ii) Change directory to temp restore location!

[zimbra@mail bin]# cd /home/dar_restore

iii) Restore user's store backup and db to tempdir.
Store:

• Make sure you have enough room on the partition, as this grabs all their mail not just the folder you are restoring.
• You need just the full and desired (usually latest) diff.
• These commands restore to the current directory. To restore elsewhere, use -R option.

[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g store/0/8/msg
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g store/0/8/msg

Databases:

• Necessary only if you do not have mysqldump's of db's.
• If you don't have sql dumps of users' dbs, extract the backups of the db files.
• You will need the user's db directory (e.g. db/data/mboxgroup8), the mysql db directory, and any files in db/data whose names begin with "ib".
• You need just the full and desired (usually latest) diff.
• These commands restore to the current directory. To restore elsewhere, use -R option.

[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/mboxgroup8
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/mboxgroup8
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/mysql
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/mysql
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/ibdata1
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/ibdata1
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/ib_logfile0
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/ib_logfile0
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/ib_logfile1
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/ib_logfile1

Use the correct mboxgroup for the user, in this case mboxgroup8.

5) Recreate old db, in order to grab message_ids. There are 2 ways, in order of preference:
a) Schedule regular mysql dumps of user mailboxes, perhaps incorporated into the backup script itself. Then restore the dump to a temp db. (MySQL, including the one in Zimbra, contains an empty "test" db, by default. We will use that.) OR
b) Use a different MySQL server to query the user's old db.

a) If you have a sql dump, you can restore to the test db:
Set the zimbra environment...

su - zimbra
source ~/bin/zmshutil ; zmsetvars

If you have the processed databases, using the zmDBbac.sh script, and don't have the raw mysql dumps, convert the dump back to raw If you have the raw DB already, skip this step.:

[zimbra@mail tmp]$ /usr/local/sbin/zmDBbac.sh makeraw /backup/sync/db_dumps/mailboxes/8.sql 8-raw.sql

then import the user's db backup into the 'test' db:

[zimbra@mail tmp]$ mysql test < 8-raw.sql

You should not have to restart Zimbra if you do sql dumps, so this method is easier all around, which argues for including it in the backup script, as I have done here: Adding per user backup and restore. (Dumping all the dbs is discussed here.)

OR

b)If you are using a different MySQL server for this step, stop the mysqld service, mv /var/lib/mysql /var/lib/mysql.bak, create a new /var/lib/mysql directory, copy from backup/db/data the old db directory, the mysql directory, all ib_logfile and ibdata files, chown -R mysql:mysql /var/lib/mysql, and restart the mysql service. Copy the user's backup store to a suitable location for the next step. You may need to add this line to the [mysqld] section of my.cnf:

innodb_log_file_size      = 104857600

6) Run the following script that grabs the index_ids for the desired folder, does a find for the message files based on those index_ids, then copies all of the messages into a RESTORE directory in the current directory. "It works for me", but I only tried it on a small folder.

Run in a directory you have write privileges to, on a partition with enough space. Example (for user "8"):
If you are restoring the user's db into the 'test' db on your live Zimbra server...

/path/to/script/recreate_Folder.sh test 2 /home/msg

If you are using the 5b method instead, and restoring the user to another Zimbra server:

/path/to/script/recreate_Folder.sh 8 2 /home/msg

In either case, "8" being the user's mailboxId from Step 2 and "2" being the folder Id from Step 3.

#!/bin/bash
# USAGE:    -h or --help for help & usage.
#           recreate_Folder.sh mailbox folderId /path/to/folder/containing/msg
#           e.g
#           recreate_Folder.sh test 2 /backup/store/0/8/msg
#           or, if not using the 'test' db, and restoring to a different server
#           recreate_Folder.sh 8 2 /backup/store/0/8/msg
mailboxId="$1"
folderId="$2"
BackupStore="$3"
RESTORE=/tmp/RESTORE     # path to location of restored message files. Set to a location with enough room.

function checkUser {
    if [ $USER != "zimbra" ] ; then
       echo "Run script as the zimbra user!"
       exit 1
    fi
    checkVariables
}

function checkVariables {
    if [ -z "$mailboxId" ] || [ -z "$folderId" ] || [ -z "BackupStore" ] ; then
       show_help
       exit 1
    else
       checkDirectory
    fi
}

function checkDirectory {
    if ! [ -d "$BackupStore" 2>/dev/null ] ; then
       echo
       echo "Message directory does not exist!"
       show_help
       exit 0
    elif [ -d ./RESTORE 2>/dev/null ] ; then
       echo
       echo "RESTORE folder already exists!"
       show_help
       exit 0
    else
       mboxgroupDB
    fi
}

function mboxgroupDB {
if (("$mailboxId" > 100)) ; then
    if ((`echo "$mailboxId" | grep -o '..$' | sed 's/0*//'` > 9)) ; then
        mboxgroup=`echo "$mailboxId" | grep -o '..$'`
    else
        mboxgroup=`echo "$mailboxId" | grep -o '.$'`
    fi
else
    mboxgroup="$mailboxId"
fi
addMessages
}

function addMessages {
source ~/bin/zmshutil ; zmsetvars
if [ -f /tmp/Folder_Msgs ] ; then
  rm /tmp/Folder_Msgs
fi
mkdir -p "$RESTORE"/msg/0
cd "$BackupStore" 
if [ "$mailboxId" = "test" ] ; then
   mysql -N -e "SELECT index_id FROM mail_item WHERE folder_id="$folderId" AND mailbox_id="$mailboxId test | grep -v + > /tmp/Folder_Ids
else
   mysql -N -e "SELECT index_id FROM mail_item WHERE folder_id="$folderId" AND mailbox_id="$mailboxId mboxgroup$mboxgroup | grep -v + > /tmp/Folder_Ids
fi
while read index_id ; do
  find "$BackupStore" -print | grep "/$index_id-" >> /tmp/Folder_Msgs
done < /tmp/Folder_Ids
while read message ; do
  cp -p "$message" "$RESTORE"/msg/0
done < /tmp/Folder_Msgs
}

function show_help {
    echo
    echo "USAGE:    recreate_Folder.sh test folderId /path/to/folder/containing/msg"  OR
    echo "USAGE:    recreate_Folder.sh mailboxId folderId /path/to/folder/containing/msg"
    echo "          e.g"
    echo "          recreate_Folder.sh test 2 /home/msg"
    echo "          recreate_Folder.sh 8 2 /home/msg"
    echo "          -h or --help for help & usage."
    echo
    echo
}

case $1 in
-H | -h | --help)
        show_help
;;
*)
        checkUser
;;
esac
exit 0

7) Create a script to copy the "Date" header in every message into a new "X-Zimbra-Received" header before you do the import. The messages will be imported as if they'd been received when the message was sent rather than when they were imported.

[zimbra@mail ~]$ /path/to/add_received.sh /path/to/restored/msg

The path should be to the msg directory within the RESTORE path from the previous step.

#!/bin/bash
# USAGE:    -h or --help for help & usage.
#           add_received.sh path/to/folder/containing/msg
#           e.g
#           add_received.sh /tmp/RESTORE/msg

BackupStore="$1"

function checkDirectory {
    if [ -d "$BackupStore" 2>/dev/null ] ; then
       addReceived
    else
       echo
       echo "Directory does not exist!"
       show_help
       exit 0
    fi
}

function addReceived {
    for d in `ls -A -1 "$BackupStore"` 
      do
        HASMSG=`ls -A -1 "$BackupStore/$d"/*.msg | head -1 2>/dev/null`
        if [ -z $HASMSG ] ; then
           echo
           echo "Specified directory has no .msg files!"
           show_help
        else
            for i in `ls -A -1 "$BackupStore/$d"/*.msg`
               do
                   if ! grep "X-Zimbra-Received" $i 1>/dev/null ; then  
                        sed -i /^Date/p $i
                        sed -i '0,/^Date/s/^Date/X-Zimbra-Received/' $i
                   fi
               done
        fi
      done
}

function show_help {
    echo
    echo "USAGE:    add_received.sh /path/to/folder/containing/msg"
    echo "          e.g"
    echo "          add_received.sh /tmp/RESTORE/msg"
    echo "          -h or --help for help & usage."
    echo
    echo
}

case $1 in
-H | -h | --help)
        show_help
;;
*)
        checkDirectory
;;
esac
exit 0

8) Restore their folder messages to "Restore_Temp". As the Zimbra user:

[zimbra@mail ~]$ zmprov
prov> sm user@somewhere.com
mailbox: user@somewhere.com, size: 26.29 KB, messages: 12, unread: 4
mbox user@somewhere.com>addMessage Restore_Temp /tmp/RESTORE/msg/0

The path should be msg/0 within the RESTORE path of previous steps. (This msg/0 subfolder was added purely to allow header processing of multiple message folders.)

9) The user then can either just rename the Restore_Temp folder in their client, or compare their messages in the Restore_Temp folder in their email client vs their existing folder, remove doubles, and move the remaining messages from the Restore_Temp folder back to the desired folder, and delete the Restore_Temp folder.

10) If you used the 'test' db method, drop and recreate the empty test db, for the next time:
As Zimbra user:

source ~/bin/zmshutil ; zmsetvars
~/mysql/bin/mysqladmin -S $mysql_socket drop test
~/mysql/bin/mysqladmin -S $mysql_socket create test

[blessendor]

Thanks authors for their script!
I will be test it today.

But for now I read script row by row and I think you can do some changes in the code for users like me: I don't have a real some data folders in the /opt/zimbra - just only sym links.

So, can you add in your script next VARs (if exist settings or "" for ignore):

ZM_STORE_DIR="" # Type your real path to message "store" folder, default - blank, that mean "/opt/zimbra/store"
ZM_DATA_DIR="" # Same logic as above
ZM_DB_DIR="" # Same logic as above

ZM_BACKUP_DIR="" # Means standard /opt/zimbra/backup dir if it will be used by your script or can be ignored, because /opt/zimbra/backup is empty (for me).

For my server this settings are:

ZM_STORE_DIR="/mnt/RAID10/store"
ZM_DATA_DIR="/mnt/RAID10/data"
ZM_DB_DIR="/mnt/RAID10/db"
ZM_BACKUP_DIR="/mnt/RAID10/backup"

Sorry for my English. I hope you understood me.

[LaFong]

I don't have a real some data folders in the /opt/zimbra - just only sym links.
So, can you add in your script next VARs (if exist settings or "" for ignore):

ZM_STORE_DIR="" # Type your real path to message "store" folder, default - blank, that mean "/opt/zimbra/store"
ZM_DATA_DIR="" # Same logic as above
ZM_DB_DIR="" # Same logic as above

I'm not sure it's worth updating the script for all these separate volumes, it's kinda old school. You could make an argument for the store directory. The data and db directories are not that large.

ZM_BACKUP_DIR="" # Means standard /opt/zimbra/backup dir if it will be used by your script or can be ignored, because /opt/zimbra/backup is empty (for me).

This directory is used by the Network Edition, not FOSS. (Mine actually does have old ldap config backups, total size 1MB). It would not be worth symlinking or making a separate volume, unless you plan on upgrading to the NE version.

For my server this settings are:

ZM_STORE_DIR="/mnt/RAID10/store"
ZM_DATA_DIR="/mnt/RAID10/data"
ZM_DB_DIR="/mnt/RAID10/db"
ZM_BACKUP_DIR="/mnt/RAID10/backup"

You could just modify the existing rsync commands like:

nice -19 $RSYNC_BIN $RSYNC_OPTS $ZM_HOME /mnt/RAID10 $SYNC_DIR

This will create a RAID10 directory within your SYNC_DIR containing your store, db, and data.

If there are volumes on RAID10 you want to exclude, create the /opt/zimbra/excludes file containing just their directory names, i.e. instead of "/mnt/RAID10/unwanted", just put "unwanted". One line per excluded volume. Then modify the RSYNC_OPTS line:

RSYNC_OPTS="-aHK --delete --exclude=*.pid --delete-excluded --exclude-from=/opt/zimbra/excludes" # leave these unless you are sure you need something else

[borngunners]

I am trying to follow the procedure on a backup server. I am confuse with number 4, so I skipped it because I am doing it on a new backup server that does not have any email. I just use it to store my backup email even though zimbra is installed on it. When I try to do number 5 as a zimbra user, this is what I got:

zimbra@mail:/var/tmp/restore$ recreate_inbox.sh test@account.mail.com /opt/zimbra/store/0/23/msg
bash: recreate_inbox.sh: command not found
zimbra@mail:/var/tmp/restore$ exit

I am not sure if I am running the script the right way.

thanks

You don't have an rsync directory, or it's already been overwritten (removing the accidentally deleted emails)?

Ugh. The thing got more complicated the more I thought about it. I hope I haven't forgotten anything else.

I have not thoroughly tested this procedure, use at your own risk.

You can restore her Inbox messages by:

1) Optional: Delete or move messages that would get doubled. If the user has messages in their Inbox which are also in the Inbox backup, those messages will get doubled when you import the Inbox backup. It is probably easiest if the user just moves all of their Inbox messages into a backup folder within Zimbra, or other client.

2) Find the user's mailboxId:

[zimbra@mail bin]# zmprov getMailboxInfo user@domain.com

mailboxId: 8
quotaUsed: 12350

Note all the paths below which use the mailboxId, "8", in the path. Change that to the desired user's mailboxId.

3a) If the rsync directory exists and has all required emails, copy the user's backup store to a new location, for the next step.

[zimbra@mail ~]$ cp -pr /backup/sync/store/0/8/msg /home

3b) If the rsync directory does not exist, or is missing the required emails, restore the dar files to a new directory:
i) Create temp restore directory:

[zimbra@mail bin]# mkdir /home/restore

ii) Change directory to temp restore location!

[zimbra@mail bin]# cd /home/restore

iii) Restore user's folder backup and db to tempdir:

[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g store/0/8/msg
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04132010_DIFF -w -g store/0/8/msg
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g store/0/8/msg

If you don't have sql dumps of users' dbs, extract the backups:

[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04122010_FULL -g db/data/mboxgroup8
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04132010_DIFF -w -g db/data/mboxgroup8
[zimbra@mail restore]# dar -x /Backup/zimbra_dars/zimbrabackup_04142010_DIFF -w -g db/data/mboxgroup8

Otherwise, extract or retrieve the dump for that user.
Make sure you have enough room on the partition, as this grabs all their mail not just the Inbox.

4) Recreate old db, in order to grab message_ids. There are 3 ways, in order of preference:
a) Schedule regular mysql dumps of user mailboxes, perhaps incorporated into the backup script itself. Then restore the dump to a temp db.
b) Use a different MySQL server to query the user's old db.
c) Use the Zimbra server.

a) If you have a sql dump, you can restore to the test db:

mysql test <mboxgroup8_04142010.sql

You should not have to restart Zimbra if you do sql dumps, so this method is easier all around, which argues for including it in the backup script. (Dumping all the dbs is discussed here.) Change the script below to use 'test' instead of 'mboxgroup$mailboxId' if you use the test db method.

OR

b)If you are using a different MySQL server for this step, just create an empty 'mboxgroup8' db (in this case), stop the mysqld service, copy the old db directory over it, and restart the service. You will need to adjust the recreate_Inbox.sh script below to not grab the mail files, but just generate the Inbox message_ids. This can be done by commenting out the find and cp lines. Then, on the Zimbra server, copy the Inbox_Ids file to /tmp, change the script by commenting out the mysql line, and uncommenting the find and cp lines.

OR

c)Backup existing and restore the old user's mailbox db, retrieve message_ids, restore existing db. Any messages that arrive for the user during this period will be lost when the existing db is restored. Thus, this is the least preferred method.

/opt/zimbra/bin/zmcontrol stop
cp -pr /opt/zimbra/db/data/mboxgroup8 /home
cp -pr /home/restore/db/data/mboxgroup8/* /opt/zimbra/db/data/mboxgroup8
/opt/zimbra/bin/zmcontrol start

Run the script in the next step, then restore the existing db:

/opt/zimbra/bin/zmcontrol stop
rm -fr /opt/zimbra/db/data/mboxgroup8
mv /home/mboxgroup8 /opt/zimbra/db/data
/opt/zimbra/bin/zmcontrol start

5) Run the following script that grabs the index_ids for the Inbox, does a find for the message files based on those index_ids, then copies all of the messages into an INBOX directory in the current directory. "It works for me", but I only tried it on a small Inbox.

Run as the Zimbra user, in a directory you have write privileges to, on a partition with enough space.

recreate_Inbox.sh joe@company.com /home/restore/store/0/8/msg

#!/bin/sh
# USAGE:    -h or --help for help & usage.
#           recreate_Inbox.sh user@domain.com path/to/folder/containing/msg/directory
#           e.g
#           recreate_Inbox.sh joe@company.com /home/restore/store/0/8/msg

User="$1"
BackupStore="$2"
PWD=`pwd`
INBOX="$PWD"/INBOX

function checkUser {
    if [ -n "$User" 2>/dev/null ] ; then
       checkDirectory
    else
       echo
       echo "No User Specified!"
       show_help
       exit 0
    fi
}

function checkDirectory {
    if ! [ -d "$BackupStore" 2>/dev/null ] ; then
       echo
       echo "Message directory does not exist!"
       show_help
       exit 0
    elif [ -d ./INBOX 2>/dev/null ] ; then
       echo
       echo "INBOX directory already exists!"
       show_help
       exit 0
    else
       addMessages
    fi
}

function addMessages {
if [ -f /tmp/Inbox_Msgs ] ; then
  rm /tmp/Inbox_Msgs
fi
mkdir INBOX
cd "$BackupStore"
mailboxId=`zmprov getMailboxInfo "$User" | grep mailboxId | awk '{print $2}'`
mysql -N -e "SELECT index_id FROM mail_item WHERE folder_id=2 mboxgroup$mailboxId" | grep -v + > /tmp/Inbox_Ids
while read index_id ; do
  find "$BackupStore" -print | grep "/$index_id-" >> /tmp/Inbox_Msgs
done < /tmp/Inbox_Ids
while read message ; do
  cp -p "$message" "$INBOX"
done < /tmp/Inbox_Msgs
}

function show_help {
    echo
    echo "USAGE:    recreate_Inbox.sh user@domain.com path/to/folder/containing/msg/directory"
    echo "          e.g"
    echo "          recreate_Inbox.sh joe@company.com /home/restore/store/0/8/msg"
    echo "          -h or --help for help & usage."
    echo
    echo
}

case $1 in
-H | -h | --help)
        show_help
;;
*)
        checkUser
;;
esac
exit 0

6) Create a script to copy the "Date" header in every message into a new "X-Zimbra-Received" header before you do the import. The messages will be imported as if they'd been received when the message was sent rather than when they were imported. Here is a rough script which I have not thoroughly debugged. Use at your own risk. Do NOT use on the live Zimbra store, or on your pristine backup. Check the results before trying to import messages.

[zimbra@mail ~]$ add_received.sh /tmp/INBOX

(Change target path to suit.)

#!/bin/sh
# USAGE:    -h or --help for help & usage.
#           add_received.sh path/to/folder/containing/msg/directory
#           e.g
#           add_received.sh /home/msg

BackupStore="$1"

function checkDirectory {
    if [ -d "$BackupStore" 2>/dev/null ] ; then
       addReceived
    else
       echo
       echo "Directory does not exist!"
       show_help
       exit 0
    fi
}

function addReceived {
    for d in `ls -A -1 "$BackupStore"` 
      do
        HASMSG=`ls -A -1 "$BackupStore/$d"/*.msg | head -1 2>/dev/null`
        if [ -z $HASMSG ] ; then
           echo
           echo "Specified directory has no .msg files!"
           show_help
        else
            for i in `ls -A -1 "$BackupStore/$d"/*.msg`
               do
                   if ! grep "X-Zimbra-Received" $i 1>/dev/null ; then  
                        sed -i /^Date/p $i
                        sed -i '0,/^Date/s/^Date/X-Zimbra-Received/' $i
                   fi
               done
        fi
      done
}

function show_help {
    echo
    echo "USAGE:    add_received.sh path/to/folder/containing/msg/files"
    echo "          e.g"
    echo "          add_received.sh /home/msg_backup"
    echo "          -h or --help for help & usage."
    echo
    echo
}

case $1 in
-H | -h | --help)
        show_help
;;
*)
        checkDirectory
;;
esac
exit 0

7) Restore their Inbox. As the Zimbra user:

[zimbra@mail ~]$ zmprov
prov> sm joe@company.com
mailbox: joe@company.com, size: 26.29 KB, messages: 12, unread: 4
mbox joe@company.com>addMessage INBOX /tmp/INBOX

8) The user then compares her Inbox messages in the backup folder vs her Inbox, removes doubles, and moves the remaining messages from the temp folder back to the Inbox, and deletes the temp folder.

9) If you used the 'test' db method, drop and recreate the empty test db, for the next time:

source ~/bin/zmshutil ; zmsetvars
~/mysql/bin/mysqladmin -S $mysql_socket -u root --password=$mysql_root_password drop test
~/mysql/bin/mysqladmin -S $mysql_socket -u root --password=$mysql_root_password create test

You'll have to figure out any path differences, etc, for your situation.

This procedure could be adapted to other folders. Their ids may be determined as follows:

mysql mboxgroup$mailboxId -N -e "SELECT id, name FROM mail_item WHERE type=1" | grep -v +

e.g.

mysql mboxgroup8 -N -e "SELECT id, name FROM mail_item WHERE type=1" | grep -v +

Somehow, Zimbra mixes "id" and "folder_id". You use the id values for folder_id lookup of index_ids. I have no idea why.

[LaFong]

I am trying to follow the procedure on a backup server. I am confuse with number 4, so I skipped it because I am doing it on a new backup server that does not have any email. I just use it to store my backup email even though zimbra is installed on it.

When I try to do number 5 as a zimbra user, this is what I got:

zimbra@mail:/var/tmp/restore$ recreate_inbox.sh test@account.mail.com /opt/zimbra/store/0/23/msg
bash: recreate_inbox.sh: command not found
zimbra@mail:/var/tmp/restore$ exit

You don't have the path. So run it with it included:

/path/to/script/recreate_inbox.sh test@account.mail.com /opt/zimbra/store/0/23/msg