Full Server Backup and Restore of “open source version”

[mubley]

It looks like the script is not getting the mailbox id for this user correctly, so it is backing up the whole index directory and the whole store directory (all mailboxes).
Please do the following:

/opt/zimbra/bin/mysql

This will put you in an interactive mysql session. Then type the following:

connect zimbra

This selects the zimbra database. Next, type the following:

select * from mailbox where comment = 'rwss920@redwingshoes.ca';

This searches the zimbra database for any records whose comment field contains the email address in question. My guess is that this will return "Empty set". Is that correct? When finished with the interactive mysql session, type:

exit

If the query returned "Empty set", then we'll need to come up with a different way to determine the mailbox ID.

Originally Posted by rmvg

32K ./index/7/index/0
36K ./index/7/index
40K ./index/7
8.0K ./index/52/index/0
12K ./index/52/index
16K ./index/52
(snip)

[rmvg]

Thank u alot for the quick reponses

Yep u are right came back empty set

[root@shoemasters rwss920@redwingshoes.ca]# su zimbra
[zimbra@shoemasters rwss920@redwingshoes.ca]$ /opt/zimbra/bin/mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1063 to server version: 4.1.10a-standard-log

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> connect zimbra
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Connection id: 1064
Current database: zimbra

mysql> select * from mailbox where comment = 'rwss920@redwingshoes.ca';
Empty set (0.01 sec)

mysql> exit
Bye

[mubley]

I've added some new code to get the mailbox ID. Please try the following script. The copy commands are commented out, so it won't actually back any mailboxes up, but it should output some info for each mailbox. Let it run for a few mailboxes (especially rwss920@redwingshoes.ca) and then kill it with Ctrl-C. Copy the output into your reply so I can make sure it looks OK.

#!/bin/bash

export time=`date +%Y-%m-%d_%H-%M-%S`
# Change this to the location where you want to store your backups
export backup_dir=/media/NO_NAME/zimbra/hot_backup/$time
mkdir -p $backup_dir
export zimbra_dir=/opt/zimbra
export index_dir=$zimbra_dir/index
export store_dir=$zimbra_dir/store

# We need mysqldump.  If it doesn't exist,
# copy mysql and modify it to call mysqldump.
if [ ! -e $zimbra_dir/bin/mysqldump ]
then
cp $zimbra_dir/bin/mysql $zimbra_dir/bin/mysqldump
sed -i 's|/bin/mysql |/bin/mysqldump |g' $zimbra_dir/bin/mysqldump
fi

# Get a list of all accounts and execute
# the contents of the for-loop for each account.
for i in `$zimbra_dir/bin/zmprov gaa`
do

# Beginning of for-loop

echo
echo --------------------------------------------------------
echo Backing up mailbox $i
echo --------------------------------------------------------

# Set zimbraAccountStatus to "maintenance"
#$zimbra_dir/bin/zmprov ma $i zimbraAccountStatus maintenance

# Verify that zimbraAccountStatus got set to "maintenance"
#$zimbra_dir/bin/zmprov ga $i |grep zimbraAccountStatus

# Use email address to get zimbra_id
zimbraId=`$zimbra_dir/bin/zmprov ga $i |grep zimbraId |sed 's|zimbraId: ||g'`
echo zimbraId is $zimbraId
zimbra_id=`$zimbra_dir/bin/mysql -e "use zimbra; select id from mailbox where account_id = '$zimbraId'"`
echo zimbra_id is $zimbra_id
zimbra_id=`echo $zimbra_id | sed 's|id ||g'`
echo zimbra_id is $zimbra_id

# Copy all files in $index_dir/0/$zimbra_id
#mkdir -p $backup_dir/$i/index/
#cp -r $index_dir/0/$zimbra_id/* $backup_dir/$i/index/
#du $backup_dir/$i/index/

# Copy all files in $store_dir/0/$zimbra_id
#mkdir -p $backup_dir/$i/store/
#cp -r $store_dir/0/$zimbra_id/* $backup_dir/$i/store/
#du $backup_dir/$i/store/

# Copy the user's MySQL database
#$zimbra_dir/bin/mysqldump mailbox$zimbra_id > $backup_dir/$i/mailbox$zimbra_id.sql
#ls -al $backup_dir/$i/mailbox$zimbra_id.sql

# Set zimbraAccountStatus to "active"
#$zimbra_dir/bin/zmprov ma $i zimbraAccountStatus active

# Verify that zimbraAccountStatus got set to "active"
#$zimbra_dir/bin/zmprov ga $i |grep zimbraAccountStatus

# End of for-loop
done

# All mailboxes have been backed up.
# Backup Zimbra master database and LDAP accounts database.
# First, set all accounts to maintenance.
echo
echo --------------------------------------------------------
echo Setting all accounts to maintenance mode to finish backup.
echo --------------------------------------------------------
for i in `$zimbra_dir/bin/zmprov gaa`
do
$zimbra_dir/bin/zmprov ma $i zimbraAccountStatus maintenance
done

echo
echo --------------------------------------------------------
echo Backing up Zimbra master database
echo --------------------------------------------------------
$zimbra_dir/bin/mysqldump zimbra > $backup_dir/zimbra.sql
ls -al $backup_dir/zimbra.sql

echo
echo --------------------------------------------------------
echo Backing up LDAP accounts database
echo --------------------------------------------------------
su - zimbra -c "$zimbra_dir/bin/zmslapcat /tmp/"
cp /tmp/ldap.bak $backup_dir/
rm /tmp/ldap.bak
ls -al $backup_dir/ldap.bak

echo
echo --------------------------------------------------------
echo Setting all accounts to active mode.
echo --------------------------------------------------------
for i in `$zimbra_dir/bin/zmprov gaa`
do
$zimbra_dir/bin/zmprov ma $i zimbraAccountStatus active
done

echo
echo --------------------------------------------------------
echo Backup is complete!
echo --------------------------------------------------------

[mubley]

I've updated the script and added a check to ensure that the mailbox id is valid.

DISCLAIMERS:
-I give no guarantees that this script will work for you and cannot be held liable for any damages that may occur.
-Use at your own risk!
-If your Zimbra server breaks, you get to keep both pieces.
-This script has NOT been tested in a production environment.
-This script has not had much testing at all.
-I have not yet verified that I can perform a successful restore from the backups that are produced by this script.
-The restore script is left as an exercise to the reader.

#!/bin/bash

export time=`date +%Y-%m-%d_%H-%M-%S`

########################################
# Modify the following variables to suit your installation
export backup_dir=/media/usbdisk/zimbra/hot_backup/$time
export zimbra_dir=/opt/zimbra
export index_dir=$zimbra_dir/index
export store_dir=$zimbra_dir/store
# Do not modify anything below this point
########################################

# Create the backup directory
mkdir -p $backup_dir

# We need mysqldump.  If it doesn't exist,
# copy mysql and modify it to call mysqldump.
if [ ! -e $zimbra_dir/bin/mysqldump ]
then
cp $zimbra_dir/bin/mysql $zimbra_dir/bin/mysqldump
sed -i 's|/bin/mysql |/bin/mysqldump |g' $zimbra_dir/bin/mysqldump
fi

# Get a list of all accounts and execute
# the contents of the for-loop for each account.
for i in `$zimbra_dir/bin/zmprov gaa`
do

# Beginning of for-loop

echo
echo --------------------------------------------------------
echo $i
echo --------------------------------------------------------

# Use email address to get zimbra_id
zimbraId=`$zimbra_dir/bin/zmprov ga $i |grep zimbraId |sed 's|zimbraId: ||g'`
#echo zimbraId is $zimbraId
zimbra_id=`$zimbra_dir/bin/mysql -e "use zimbra; select id from mailbox where account_id = '$zimbraId'"`
#echo zimbra_id is $zimbra_id
zimbra_id=`echo $zimbra_id | sed 's|id ||g'`
#echo zimbra_id is $zimbra_id

# Check to make sure we got a valid zimbraId
if [ "$zimbra_id" = "" ]; then
echo $i has no mailbox.

else

# Set zimbraAccountStatus to "maintenance"
$zimbra_dir/bin/zmprov ma $i zimbraAccountStatus maintenance

# Verify that zimbraAccountStatus got set to "maintenance"
$zimbra_dir/bin/zmprov ga $i |grep zimbraAccountStatus

# Copy all files in $index_dir/0/$zimbra_id
mkdir -p $backup_dir/$i/index/
cp -r $index_dir/0/$zimbra_id/* $backup_dir/$i/index/
du $backup_dir/$i/index/

# Copy all files in $store_dir/0/$zimbra_id
mkdir -p $backup_dir/$i/store/
cp -r $store_dir/0/$zimbra_id/* $backup_dir/$i/store/
du $backup_dir/$i/store/

# Copy the user's MySQL database
$zimbra_dir/bin/mysqldump mailbox$zimbra_id > $backup_dir/$i/mailbox$zimbra_id.sql
ls -al $backup_dir/$i/mailbox$zimbra_id.sql

# Set zimbraAccountStatus to "active"
$zimbra_dir/bin/zmprov ma $i zimbraAccountStatus active

# Verify that zimbraAccountStatus got set to "active"
$zimbra_dir/bin/zmprov ga $i |grep zimbraAccountStatus

# End of if statement
fi

# End of for-loop
done

# All mailboxes have been backed up.
# Backup Zimbra master database and LDAP accounts database.
# First, set all accounts to maintenance.
echo
echo --------------------------------------------------------
echo Setting all accounts to maintenance mode to finish backup.
echo --------------------------------------------------------
for i in `$zimbra_dir/bin/zmprov gaa`
do
$zimbra_dir/bin/zmprov ma $i zimbraAccountStatus maintenance
done

echo
echo --------------------------------------------------------
echo Backing up Zimbra master database
echo --------------------------------------------------------
$zimbra_dir/bin/mysqldump zimbra > $backup_dir/zimbra.sql
ls -al $backup_dir/zimbra.sql

echo
echo --------------------------------------------------------
echo Backing up LDAP accounts database
echo --------------------------------------------------------
su - zimbra -c "$zimbra_dir/bin/zmslapcat /tmp/"
cp /tmp/ldap.bak $backup_dir/
rm /tmp/ldap.bak
ls -al $backup_dir/ldap.bak

echo
echo --------------------------------------------------------
echo Setting all accounts to active mode.
echo --------------------------------------------------------
for i in `$zimbra_dir/bin/zmprov gaa`
do
$zimbra_dir/bin/zmprov ma $i zimbraAccountStatus active
done

echo
echo --------------------------------------------------------
echo Backup is complete!
echo --------------------------------------------------------

[rmvg]

here is the output does this tell u anything? opps i let it run until setting all accounts to maintance mode and then killed it. If i run it again it should shutdown all account and then reactivate all account right?



--------------------------------------------------------
Backing up mailbox rwss918@redwingshoes.ca
--------------------------------------------------------
zimbraId is f4e9990c-602f-11da-8712-db2bbc1d5b2d
zimbra_id is id 6
zimbra_id is 6

--------------------------------------------------------
Backing up mailbox rwss919@redwingshoes.ca
--------------------------------------------------------
zimbraId is db62de6b-602f-11da-8712-db2bbc1d5b2d
zimbra_id is id 9
zimbra_id is 9

--------------------------------------------------------
Backing up mailbox rwss920@redwingshoes.ca
--------------------------------------------------------
zimbraId is f916e3ed-6191-11da-a743-49da4415eb96
zimbra_id is id 8
zimbra_id is 8

--------------------------------------------------------
Backing up mailbox rwss923@redwingshoes.ca
--------------------------------------------------------
zimbraId is 6633c7ec-61c2-11da-97e9-15b7c2d6f4a0
zimbra_id is id 10
zimbra_id is 10

--------------------------------------------------------
Backing up mailbox rwss924@redwingshoes.ca
--------------------------------------------------------
zimbraId is 1cc88127-61e8-11da-97e9-15b7c2d6f4a0
zimbra_id is id 14
zimbra_id is 14

--------------------------------------------------------
Backing up mailbox rwss925@redwingshoes.ca
--------------------------------------------------------
zimbraId is 3680d318-61e8-11da-97e9-15b7c2d6f4a0
zimbra_id is id 37
zimbra_id is 37

--------------------------------------------------------
Backing up mailbox sergio@redwingshoes.ca
--------------------------------------------------------
zimbraId is 922ad023-6e71-11da-8375-bdaeeae70a87
zimbra_id is id 59
zimbra_id is 59

--------------------------------------------------------
Backing up mailbox admin@sculpturaldesign.ca
--------------------------------------------------------
zimbraId is f9f5eef5-6eae-11da-8375-bdaeeae70a87
zimbra_id is id 60
zimbra_id is 60

--------------------------------------------------------
Backing up mailbox keatingc@sculpturaldesign.ca
--------------------------------------------------------
zimbraId is 941d211a-6eaf-11da-8375-bdaeeae70a87
zimbra_id is id 61
zimbra_id is 61

--------------------------------------------------------
Backing up mailbox salloumb@sculpturaldesign.ca
--------------------------------------------------------
zimbraId is 2c270df6-6eaf-11da-8375-bdaeeae70a87
zimbra_id is id 62
zimbra_id is 62

--------------------------------------------------------
Backing up mailbox acadia@shoemasters.com
--------------------------------------------------------
zimbraId is 4681769f-6192-11da-a743-49da4415eb96
zimbra_id is id 38
zimbra_id is 38

--------------------------------------------------------
Backing up mailbox admin@shoemasters.com
--------------------------------------------------------
zimbraId is adc71414-5ece-11da-8f50-0f77a03b5126
zimbra_id is id 1
zimbra_id is 1

--------------------------------------------------------
Backing up mailbox bonniedoon@shoemasters.com
--------------------------------------------------------
zimbraId is 1dd8b3e9-601d-11da-8712-db2bbc1d5b2d
zimbra_id is id 5
zimbra_id is 5

--------------------------------------------------------
Backing up mailbox bowvalley@shoemasters.com
--------------------------------------------------------
zimbraId is 8968f45d-61e6-11da-97e9-15b7c2d6f4a0
zimbra_id is id 11
zimbra_id is 11

--------------------------------------------------------
Backing up mailbox chinook@shoemasters.com
--------------------------------------------------------
zimbraId is df32efcb-61ab-11da-97e9-15b7c2d6f4a0
zimbra_id is id 21
zimbra_id is 21

--------------------------------------------------------
Backing up mailbox hill@shoemasters.com
--------------------------------------------------------
zimbraId is 7d473011-65a4-11da-8635-73705ff3e3ce
zimbra_id is id 49
zimbra_id is 49

--------------------------------------------------------
Backing up mailbox hillr@shoemasters.com
--------------------------------------------------------
zimbraId is 51bfb51a-659d-11da-8241-795b8c73ad12
zimbra_id is id 47
zimbra_id is 47

--------------------------------------------------------
Backing up mailbox king@shoemasters.com
--------------------------------------------------------
zimbraId is 523ea838-5f3e-11da-9bd3-0f252ffc7ba2
zimbra_id is id 4
zimbra_id is 4

--------------------------------------------------------
Backing up mailbox kingsway@shoemasters.com
--------------------------------------------------------
zimbraId is bcfe373f-61e6-11da-97e9-15b7c2d6f4a0
zimbra_id is id 39
zimbra_id is 39

--------------------------------------------------------
Backing up mailbox lethbridge@shoemasters.com
--------------------------------------------------------
zimbraId is 0aebb592-61e7-11da-97e9-15b7c2d6f4a0
zimbra_id is id 15
zimbra_id is 15

--------------------------------------------------------
Backing up mailbox londonderry@shoemasters.com
--------------------------------------------------------
zimbraId is 1b34744d-6030-11da-8712-db2bbc1d5b2d
zimbra_id is id 7
zimbra_id is 7

--------------------------------------------------------
Backing up mailbox marketmall@shoemasters.com
--------------------------------------------------------
zimbraId is 0ec8539e-6192-11da-a743-49da4415eb96
zimbra_id is id 40
zimbra_id is 40

--------------------------------------------------------
Backing up mailbox northlands@shoemasters.com
--------------------------------------------------------
zimbraId is d5cdfdf0-61e6-11da-97e9-15b7c2d6f4a0
zimbra_id is id 12
zimbra_id is 12

--------------------------------------------------------
Backing up mailbox quyen@shoemasters.com
--------------------------------------------------------
zimbraId is d6ea7e9b-61f3-11da-97e9-15b7c2d6f4a0
zimbra_id is id 13
zimbra_id is 13

--------------------------------------------------------
Backing up mailbox southcenter@shoemasters.com
--------------------------------------------------------
zimbraId is a44b475e-61e6-11da-97e9-15b7c2d6f4a0
zimbra_id is id 42
zimbra_id is 42

--------------------------------------------------------
Backing up mailbox southgate@shoemasters.com
--------------------------------------------------------
zimbraId is 4e03cf1e-6030-11da-8712-db2bbc1d5b2d
zimbra_id is id 43
zimbra_id is 43

--------------------------------------------------------
Backing up mailbox tdsquare@shoemasters.com
--------------------------------------------------------
zimbraId is f521dbe1-61e6-11da-97e9-15b7c2d6f4a0
zimbra_id is id 44
zimbra_id is 44

--------------------------------------------------------
Backing up mailbox andersont@ybotstudios.com
--------------------------------------------------------
zimbraId is 728ddf9b-6373-11da-8241-795b8c73ad12
zimbra_id is id 22
zimbra_id is 22

[mubley]

Yes, it looks like the new code is working properly. Try my updated script (directly above your reply). Let it finish and all accounts should then be set to active. Thanks for testing.

Originally Posted by rmvg

here is the output does this tell u anything? opps i let it run until setting all accounts to maintance mode and then killed it. If i run it again it should shutdown all account and then reactivate all account right?



--------------------------------------------------------
Backing up mailbox rwss918@redwingshoes.ca
--------------------------------------------------------
zimbraId is f4e9990c-602f-11da-8712-db2bbc1d5b2d
zimbra_id is id 6
zimbra_id is 6
(snip)

[bobby]

i know the -h means "human readable" and this is no longer relevant here , but if you run du -k it's easier to see what's big and you can do stuff like du -k | sort -n


in addition, you can zmprov your accounts yourself to reactivate them from maintenance mode:

zimbra@host $ zmprov ma [username] zimbraAccountStatus active

or you can set them all at automatically (like mubley does):
zimbra@host $ for x in `zmprov gaa`; do zmprov ma $x zimbraAccountStatus active; done

[rmvg]

thanks for the tips bobby mabey u can tell me how to get output of the script to a file somewhere.

I am using putty on a wireless connection and it keeps dieing on me so i cannot tell weather the script completed or not. I tried script /var/tmp/zimbra1 but it dies as soon as the putty term does!!!

[bobby]

how to get output of the script to a file somewhere

$ command > filename

so i cannot tell weather the script completed or not

$ nohup command &

nohup tells it to keep running even if the tty dies (and it automatically sends output to a file called nohup.out)

& brings you back to the prompt while it runs in the background

[mubley]

rmvg, have you been able to get the new script working?