would like advice on my backup script please

[xkill911]

Zimbra: Release 7.1.3_GA_3346.UBUNTU10_64 UBUNTU10_64 FOSS edition

Server: Ubuntu 10.14 64Bit

this is the backup script I have been using for a few years now and I need to take another look at it

basically what it does is

1. stop Zimbra services

2. zip up the whole /opt/zimbra/ folder to a /backup/ folder and create a tgz file with version and date

3. start Zimbra services

4. send the backup to an FTP server of your choosing

could you experts please look at my script and see if any improvements be made?

removed bad code, see below

[xkill911]

I tried my script last night on my new Ubuntu 10.04 64bit server and it failed badly. it worked perfectly on my old Ubuntu 8.04 server

not sure exactly what happened but my Zimbra was down this morning and i had to do a "zmcontrol start" to get it back working


can someone tell me if

sudo -u zimbra /opt/zimbra/bin/zmcontrol stop

and

sudo /etc/init.d/zimbra start

is still the proper commands?

thanks!

[spectra]

there is a start and stop

I would use /opt/zimbra/bin/zmcontrol stop and start

I would run the zimbra related commands as zimbra user

[Tripple]

I'm using a script found on this site. When I compare your code with the one I'm using there are some issues that might cause your problem:

# Stop Zimbra Services
echo Stopping Zimbra Services
- sudo -u zimbra /opt/zimbra/bin/zmcontrol stop
+ su - zimbra -c "/opt/zimbra/bin/zmcontrol stop"
+ sleep 15
+ ORPHANED=`ps -u zimbra -o "pid="` && kill -9 $ORPHANED

- sudo /etc/init.d/zimbra start
+ su - zimbra -c "/opt/zimbra/bin/zmcontrol start"

Open Source Edition Backup Procedure - Zimbra :: Wiki

[xkill911]

thank you very much guys!

here's the revised 100% working script

#!/bin/bash

# Zimbra Backup Script
# Requires ftp to run
# This script is intended to run from the crontab as root

# Stop Zimbra Services
echo Stopping Zimbra Services
sudo -u zimbra zmcontrol stop

# Sync to backup directory
echo Syncing the backup directory with the live install
rsync -avHK --delete /opt/zimbra/ /backup/zimbra

# Restart Zimbra Services
echo Starting the Zimbra Services...please be patient
sudo -u zimbra zmcontrol start

# Create archive of backed-up directory for offsite transfer
# cd /backup/zimbra
echo Creating the tgz archive This will take a bit
ZimbraVersion=zimbraBackup-zcs-7.1.3.UBUNTU

tar -zcf /backup/$ZimbraVersion-$(date +"%Y-%m-%d").tgz -C /backup/zimbra  .

# Transfer file to backup server using ftp
# Put FTP server details here
SERVER="ftp.yourftpserver.com"
USERNAME="yourusername"
PASSWORD="yourpassword"

echo Connecting to and uploading tgz archive to FTP server
# login to remote server
ftp -n -i <<EOF
open $SERVER
user $USERNAME $PASSWORD
lcd /backup
mput zimbraBackup-zcs-7.1.3.UBUNTU-$(date +"%Y-%m-%d").tgz
quit 
EOF

and here's the log file:

Stopping Zimbra Services
Syncing the backup directory with the live install
sending incremental file list
data/amavisd/.spamassassin/
data/amavisd/.spamassassin/bayes.mutex
data/amavisd/.spamassassin/bayes_journal
data/amavisd/.spamassassin/bayes_seen
data/amavisd/.spamassassin/bayes_toks
data/amavisd/tmp/
deleting data/amavisd/tmp/amavis-20111125T171220-14611/parts/
deleting data/amavisd/tmp/amavis-20111125T171220-14611/email.txt
deleting data/amavisd/tmp/amavis-20111125T171220-14611/
deleting data/amavisd/tmp/amavis-20111125T155410-32081/parts/
deleting data/amavisd/tmp/amavis-20111125T155410-32081/email.txt
deleting data/amavisd/tmp/amavis-20111125T155410-32081/
deleting data/amavisd/tmp/amavis-20111125T155331-32085/parts/
deleting data/amavisd/tmp/amavis-20111125T155331-32085/email.txt
deleting data/amavisd/tmp/amavis-20111125T155331-32085/
deleting data/amavisd/tmp/amavis-20111125T154459-32077/parts/
deleting data/amavisd/tmp/amavis-20111125T154459-32077/email.txt
deleting data/amavisd/tmp/amavis-20111125T154459-32077/
deleting data/amavisd/tmp/amavis-20111125T135129-32079/parts/
deleting data/amavisd/tmp/amavis-20111125T135129-32079/email.txt
deleting data/amavisd/tmp/amavis-20111125T135129-32079/
deleting data/amavisd/tmp/amavis-20111125T131401-32076/parts/
deleting data/amavisd/tmp/amavis-20111125T131401-32076/email.txt
deleting data/amavisd/tmp/amavis-20111125T131401-32076/
data/amavisd/tmp/amavis-20111125T155352-32078/
data/amavisd/tmp/amavis-20111125T155352-32078/email.txt
data/amavisd/tmp/amavis-20111125T155352-32078/parts/
data/amavisd/tmp/amavis-20111125T224704-26807/email.txt
data/amavisd/tmp/amavis-20111125T224704-26807/parts/
data/amavisd/tmp/amavis-20111126T011356-28537/email.txt
data/amavisd/tmp/amavis-20111126T011356-28537/parts/
data/amavisd/tmp/amavis-20111126T052934-23745/email.txt
data/amavisd/tmp/amavis-20111126T052934-23745/parts/
data/amavisd/tmp/amavis-20111127T153211-23402/
data/amavisd/tmp/amavis-20111127T153211-23402/email.txt
data/amavisd/tmp/amavis-20111127T153211-23402/parts/
data/amavisd/tmp/amavis-20111127T173136-11812/
data/amavisd/tmp/amavis-20111127T173136-11812/email.txt
data/amavisd/tmp/amavis-20111127T173136-11812/parts/
data/amavisd/var/amavisd.lock
data/clamav/db/
data/clamav/db/mirrors.dat
data/ldap/hdb/db/__db.002
data/ldap/hdb/db/__db.003
data/ldap/hdb/db/__db.004
data/ldap/hdb/db/__db.005
data/ldap/hdb/db/__db.006
data/ldap/hdb/db/entryCSN.bdb
data/ldap/hdb/db/id2entry.bdb
data/ldap/hdb/logs/log.0000000002
data/postfix/data/prng_exch
data/postfix/spool/active/
data/postfix/spool/defer/2/
deleting data/postfix/spool/defer/2/244DCAF4E7F
data/postfix/spool/defer/4/
deleting data/postfix/spool/defer/4/43C0CAF4E84
data/postfix/spool/defer/5/
deleting data/postfix/spool/defer/5/5EB06AF4E83
deleting data/postfix/spool/defer/5/57B0FAF4E7A
data/postfix/spool/defer/6/
data/postfix/spool/defer/6/6DC17AF4F6C
data/postfix/spool/defer/9/
deleting data/postfix/spool/defer/9/99163AF4E80
deleting data/postfix/spool/defer/9/979C1AF4E7E
data/postfix/spool/defer/A/
deleting data/postfix/spool/defer/A/AC9B9AF4E85
data/postfix/spool/defer/A/A6141AF4F6B
data/postfix/spool/defer/D/
deleting data/postfix/spool/defer/D/DFC47AF4E87
data/postfix/spool/defer/D/D239CAF4F6F
data/postfix/spool/defer/D/DED71AF4F70
data/postfix/spool/deferred/2/
deleting data/postfix/spool/deferred/2/244DCAF4E7F
data/postfix/spool/deferred/4/
deleting data/postfix/spool/deferred/4/43C0CAF4E84
data/postfix/spool/deferred/5/
deleting data/postfix/spool/deferred/5/5EB06AF4E83
deleting data/postfix/spool/deferred/5/57B0FAF4E7A
data/postfix/spool/deferred/6/
data/postfix/spool/deferred/6/6DC17AF4F6C
data/postfix/spool/deferred/9/
deleting data/postfix/spool/deferred/9/99163AF4E80
deleting data/postfix/spool/deferred/9/979C1AF4E7E
data/postfix/spool/deferred/A/
deleting data/postfix/spool/deferred/A/AC9B9AF4E85
data/postfix/spool/deferred/A/A6141AF4F6B
data/postfix/spool/deferred/D/
deleting data/postfix/spool/deferred/D/DFC47AF4E87
data/postfix/spool/deferred/D/D239CAF4F6F
data/postfix/spool/deferred/D/DED71AF4F70
data/postfix/spool/incoming/
data/tmp/
deleting data/tmp/.swatch_script.7728
data/tmp/.swatch_script.10107
data/tmp/.swatch_script.10226
data/tmp/.swatch_script.9838
data/tmp/upload/
db/data/ib_logfile0
db/data/ibdata1
db/data/mboxgroup1/mail_item.ibd
db/data/mboxgroup1/open_conversation.ibd
db/data/mboxgroup11/mail_item.ibd
db/data/mboxgroup11/open_conversation.ibd
db/data/mboxgroup16/mail_item.ibd
db/data/mboxgroup16/open_conversation.ibd
db/data/mboxgroup17/mail_item.ibd
db/data/mboxgroup17/open_conversation.ibd
db/data/mboxgroup19/mail_item.ibd
db/data/mboxgroup19/open_conversation.ibd
db/data/mboxgroup2/mail_item.ibd
db/data/mboxgroup2/mail_item_dumpster.ibd
db/data/mboxgroup2/open_conversation.ibd
db/data/mboxgroup2/tombstone.ibd
db/data/mboxgroup22/open_conversation.ibd
db/data/mboxgroup26/mail_item.ibd
db/data/mboxgroup26/open_conversation.ibd
db/data/mboxgroup27/mail_item.ibd
db/data/mboxgroup27/open_conversation.ibd
db/data/mboxgroup30/mail_item.ibd
db/data/mboxgroup30/open_conversation.ibd
db/data/mboxgroup31/mail_item.ibd
db/data/mboxgroup31/open_conversation.ibd
db/data/mboxgroup34/mail_item.ibd
db/data/mboxgroup34/open_conversation.ibd
db/data/mboxgroup36/mail_item.ibd
db/data/mboxgroup36/open_conversation.ibd
db/data/mboxgroup36/tombstone.ibd
db/data/mboxgroup37/mail_item.ibd
db/data/mboxgroup37/open_conversation.ibd
db/data/mboxgroup42/data_source_item.ibd
db/data/mboxgroup5/appointment.ibd
db/data/mboxgroup5/mail_item.ibd
db/data/mboxgroup5/open_conversation.ibd
db/data/mboxgroup54/mail_item.ibd
db/data/mboxgroup54/open_conversation.ibd
db/data/mboxgroup6/open_conversation.ibd
db/data/mboxgroup61/mail_item.ibd
db/data/mboxgroup61/open_conversation.ibd
db/data/mboxgroup8/mail_item.ibd
db/data/mboxgroup8/open_conversation.ibd
db/data/mboxgroup9/mail_item.ibd
db/data/mboxgroup9/open_conversation.ibd
db/data/zimbra/config.ibd
db/data/zimbra/jiveID.MYI
db/data/zimbra/jiveRoster.MYI
db/data/zimbra/jiveUserProp.MYD
db/data/zimbra/jiveUserProp.MYI
db/data/zimbra/mailbox.ibd
db/data/zimbra/mailbox_metadata.ibd
index/0/17/index/0/
deleting index/0/17/index/0/segments_1x
index/0/17/index/0/_61.cfs
index/0/17/index/0/segments.gen
index/0/17/index/0/segments_1y
index/0/36/index/0/
deleting index/0/36/index/0/segments_13
deleting index/0/36/index/0/_2n.cfs
deleting index/0/36/index/0/_2m.cfs
deleting index/0/36/index/0/_2l.cfs
deleting index/0/36/index/0/_2k.cfs
deleting index/0/36/index/0/_2j.cfs
deleting index/0/36/index/0/_2i.cfs
deleting index/0/36/index/0/_2h_1.del
deleting index/0/36/index/0/_2h.cfs
deleting index/0/36/index/0/_2g_1.del
deleting index/0/36/index/0/_2g.cfs
deleting index/0/36/index/0/_2f.cfs
index/0/36/index/0/_2p.cfs
index/0/36/index/0/_2q.cfs
index/0/36/index/0/_2r.cfs
index/0/36/index/0/_2s.cfs
index/0/36/index/0/segments.gen
index/0/36/index/0/segments_17
index/0/9/index/0/
deleting index/0/9/index/0/segments_e
index/0/9/index/0/_10.cfs
index/0/9/index/0/segments.gen
index/0/9/index/0/segments_f
jetty-6.1.22.z6/work/resource-cache/skinres/latest/
jetty-6.1.22.z6/work/resource-cache/skinres/latest/zimbra-0349a8776cee992cb1a472810aca3629-2955.javascript
jetty-6.1.22.z6/work/resource-cache/skinres/latest/zimbra-2e5af1a03ab2de4c72905bdd2687b76b-2955.css
jetty-6.1.22.z6/work/resource-cache/skinres/latest/zimbra-484d2417c9430405877268d0f984247c-2954.css
jetty-6.1.22.z6/work/resource-cache/skinres/latest/zimbra-60eb7583587ad7629beeb5b1ec3ac963-2954.css
jetty-6.1.22.z6/work/resource-cache/skinres/latest/zimbra-6a3ad78f02f5c2a69d2d6e07df8db0de-2955.css
jetty-6.1.22.z6/work/resource-cache/skinres/latest/zimbra-6b9c5a6073b6931f02f2aabb8cc71688-2955.css
jetty-6.1.22.z6/work/resource-cache/skinres/latest/zimbra-9f131a55e334f4add7240fda6db5c902-2955.css
jetty-6.1.22.z6/work/resource-cache/skinres/latest/zimbra-d767e37dfe5520e7d4b4db05d46028c9-3321.css
jetty-6.1.22.z6/work/resource-cache/skinres/latest/zimbra-ed1380f7ed7b02655e2de176a4bff899-2955.css
jetty-6.1.22.z6/work/resource-cache/skinres/latest/zimbra-f437548d958c81d73d0e9115bf1f3c43-2951.css
log/
log/.zmcontrol.cache
log/2011_11_27.trace.log
log/2011_11_28.trace.log
log/access_log.2011-11-27
log/access_log.2011-11-28
log/audit.log
log/audit.log.2011-11-27.gz
log/clamd.log
log/freshclam.log
log/httpd_access.log.2011-11-27
log/logswatch.pid
log/mailbox.log
log/mailbox.log.2011-11-27.gz
log/spamtrain.log
log/swatch.pid
log/zmconfigd.log
log/zmconfigd.log.1
log/zmlogprocess.state
log/zmlogswatch.out
log/zmlogswatch.out.1.gz
log/zmlogswatch.out.2.gz
log/zmlogswatch.out.3.gz
log/zmmailboxd.out
log/zmmailboxd.out.201111280000
log/zmrrdfetch-server.pid
log/zmswatch.out
log/zmswatch.out.1.gz
log/zmswatch.out.2.gz
log/zmswatch.out.3.gz
log/zmswatch.out.4.gz
log/zmswatch.out.5.gz
logger/db/data/
logger/db/data/logger.sqlitedb
logger/db/data/rrds/1-0.rrd
logger/db/data/rrds/1-1.rrd
logger/db/data/rrds/1-2.rrd
logger/db/data/rrds/1-3.rrd
logger/db/data/rrds/1-4.rrd
logger/db/data/rrds/1-5.rrd
logger/db/data/rrds/1-6.rrd
logger/db/data/rrds/1-7.rrd
logger/db/data/rrds/1-8.rrd
logger/db/data/rrds/2-0.rrd
logger/db/data/rrds/2-1.rrd
logger/db/data/rrds/2-10.rrd
logger/db/data/rrds/2-11.rrd
logger/db/data/rrds/2-2.rrd
logger/db/data/rrds/2-3.rrd
logger/db/data/rrds/2-4.rrd
logger/db/data/rrds/2-5.rrd
logger/db/data/rrds/2-6.rrd
logger/db/data/rrds/2-7.rrd
logger/db/data/rrds/2-8.rrd
logger/db/data/rrds/2-9.rrd
redolog/redo.log
store/0/1/msg/0/
store/0/1/msg/0/3062-3299.msg
store/0/1/msg/0/3063-3301.msg
store/0/11/msg/1/
store/0/11/msg/1/5948-5666.msg
store/0/11/msg/1/5949-5669.msg
store/0/11/msg/1/5950-5681.msg
store/0/11/msg/1/5951-5688.msg
store/0/11/msg/1/5952-5712.msg
store/0/16/msg/0/
store/0/16/msg/0/2720-2400.msg
store/0/17/msg/2/
store/0/17/msg/2/11002-20118.msg
store/0/17/msg/2/11003-20125.msg
store/0/17/msg/2/11004-20131.msg
store/0/17/msg/2/11005-20141.msg
store/0/17/msg/2/11006-20149.msg
store/0/17/msg/2/11007-20150.msg
store/0/17/msg/2/11009-20151.msg
store/0/17/msg/2/11010-20152.msg
store/0/17/msg/2/11011-20153.msg
store/0/17/msg/2/11012-20154.msg
store/0/17/msg/2/11013-20155.msg
store/0/17/msg/2/11014-20156.msg
store/0/19/msg/1/
store/0/19/msg/1/6583-6237.msg
store/0/2/msg/0/
store/0/2/msg/0/280-486.msg
store/0/26/msg/0/
store/0/26/msg/0/1063-1323.msg
store/0/27/msg/1/
store/0/27/msg/1/5265-5531.msg
store/0/27/msg/1/5266-5561.msg
store/0/30/msg/0/
store/0/30/msg/0/2180-2434.msg
store/0/31/msg/0/
store/0/31/msg/0/623-769.msg
store/0/31/msg/0/624-809.msg
store/0/34/msg/1/
store/0/34/msg/1/5941-5581.msg
store/0/36/msg/1/
store/0/36/msg/1/4872-4487.msg
store/0/36/msg/1/4873-4512.msg
store/0/36/msg/1/4881-4525.msg
store/0/36/msg/1/4885-4534.msg
store/0/36/msg/1/4886-4537.msg
store/0/36/msg/1/4889-4545.msg
store/0/36/msg/1/4899-4567.msg
store/0/36/msg/1/4900-4581.msg
store/0/37/msg/0/
store/0/37/msg/0/3334-5214.msg
store/0/5/msg/4/
store/0/5/msg/4/18038-17237.msg
store/0/5/msg/4/18039-17245.msg
store/0/5/msg/4/18040-17246.msg
store/0/5/msg/4/18041-17247.msg
store/0/5/msg/4/18042-17248.msg
store/0/5/msg/4/18043-17249.msg
store/0/5/msg/4/18044-17254.msg
store/0/5/msg/4/18045-17256.msg
store/0/5/msg/4/18046-17257.msg
store/0/5/msg/4/18047-17290.msg
store/0/5/msg/4/18048-17292.msg
store/0/5/msg/4/18050-17296.msg
store/0/5/msg/4/18051-17297.msg
store/0/5/msg/4/18052-17305.msg => store/0/1/msg/0/3062-3299.msg
store/0/5/msg/4/18053-17307.msg => store/0/1/msg/0/3063-3301.msg
store/0/5/msg/4/18054-17309.msg
store/0/54/msg/0/
store/0/54/msg/0/1484-1559.msg
store/0/61/msg/0/
store/0/61/msg/0/440-550.msg
store/0/8/msg/1/
store/0/8/msg/1/4853-4376.msg
store/0/8/msg/1/4854-4377.msg
store/0/8/msg/1/4855-4387.msg => store/0/37/msg/0/3334-5214.msg
store/0/8/msg/1/4856-4388.msg
store/0/8/msg/1/4857-4391.msg
store/0/9/msg/0/
store/0/9/msg/0/1874-2179.msg
store/0/9/msg/0/1875-2181.msg
store/0/9/msg/0/1876-2185.msg
store/0/9/msg/0/1877-2192.msg
store/0/9/msg/0/1878-2197.msg
store/0/9/msg/0/1879-2200.msg
store/0/9/msg/0/1880-2207.msg
store/incoming/
zmstat/
zmstat/allprocs.csv
zmstat/cpu.csv
zmstat/df.csv
zmstat/fd.csv
zmstat/io-x.csv
zmstat/io.csv
zmstat/mailboxd.csv
zmstat/mtaqueue.csv
zmstat/mysql.csv
zmstat/pop3.csv
zmstat/proc.csv
zmstat/soap.csv
zmstat/threads.csv
zmstat/vm.csv
zmstat/zmstat.out
zmstat/zmstat.out.1.gz
zmstat/zmstat.out.2.gz
zmstat/zmstat.out.3.gz
zmstat/zmstat.out.4.gz
zmstat/zmstat.out.5.gz
zmstat/zmstat.out.6.gz
zmstat/zmstat.out.7.gz
zmstat/2011-11-27/
zmstat/2011-11-27/allprocs.csv.gz
zmstat/2011-11-27/cpu.csv.gz
zmstat/2011-11-27/df.csv.gz
zmstat/2011-11-27/fd.csv.gz
zmstat/2011-11-27/io-x.csv.gz
zmstat/2011-11-27/io.csv.gz
zmstat/2011-11-27/mailboxd.csv.gz
zmstat/2011-11-27/mtaqueue.csv.gz
zmstat/2011-11-27/mysql.csv.gz
zmstat/2011-11-27/pop3.csv.gz
zmstat/2011-11-27/proc.csv.gz
zmstat/2011-11-27/soap.csv.gz
zmstat/2011-11-27/threads.csv.gz
zmstat/2011-11-27/vm.csv.gz

sent 1221627975 bytes  received 9064 bytes  22011478.18 bytes/sec
total size is 14431033287  speedup is 11.81
Starting the Zimbra Services...please be patient
Creating the tgz archive This will take a bit
Connecting to and uploading tgz archive to FTP server
Local directory now /backup

[spectra]

So, how about the restore side of things.
I think most ppl succeed at securing the zimbra folder, but they fail miserably during a restore.

Have you tested the restore process ?
For example, suppose I'm running FOSS 7.1.3 on Ubuntu 804 LTS.
Is it as simple as setup a new Ubuntu 10.4 LTS server.
Assuming both 64 bit, cant leave that detail out.
Install 713 and then shutdown and overlay the backup to /opt/zimbra.

I hope you see what I mean about the restore side of things.

I like your script alot because it's simple. Non gurus aka common folks can understand it.

So are you willing to now work on the restore side of things

[Tripple]

thank you very much guys!
here's the revised 100% working script

Might work, maybe not.
Why did you leave these lines out of the script after you stop Zimbra:

sleep 15
ORPHANED=`ps -u zimbra -o "pid="` && kill -9 $ORPHANED

This code is there to make sure there's no Zimbra process running.

[Tripple]

So, how about the restore side of things.
I think most ppl succeed at securing the zimbra folder, but they fail miserably during a restore.

Have you tested the restore process ?
For example, suppose I'm running FOSS 7.1.3 on Ubuntu 804 LTS.
Is it as simple as setup a new Ubuntu 10.4 LTS server.
Assuming both 64 bit, cant leave that detail out.
Install 713 and then shutdown and overlay the backup to /opt/zimbra.

I hope you see what I mean about the restore side of things.

I like your script alot because it's simple. Non gurus aka common folks can understand it.

So are you willing to now work on the restore side of things

Indeed I tested the restore process and needed it once on a production server as well.

Restore is very simple:
Setup your server with the same OS and the same Zimbra version
Delete /opt/zimbra
Replace /opt/zimbra with your backup
Run the Zimbra installer once more and update your ZCS

This in nothing more then a disaster recovery for a FOSS edition. If you want to restore a single account or mail you need a different procedure.

[spectra]

Very well then Tripple
One question that wasn't addressed.
Will your strategy work if my production server was Ubuntu 804LTS 64bit and I build new server with Ubuntu 10 0 4 LTS 64bit ?

From your live experience, I take it that the OS and FOSS versions were precisely the same. Yes or No

Again thanks to you and xkill911 for this resolution. Many will find it useful.

thanks

[spectra]

Yo xkil911, Tripple makes a valid point.
You need to be sure all Zimbra processes are dead.
To xkill911 and Tripple, what about mySQL ?
Don't we need to be sure that process is stopped as well too ?

XKill911 you should retest and repost with the changes suggested by Tripple.

Thanks Fellas