Zimbra offers Open Source email server software and shared calendar for Linux and the Mac
Go Back   Zimbra :: Forums > Zimbra Collaboration Suite > Administrators

Welcome to the Zimbra :: Forums!
Welcome, if you would like to post a comment please register. We also encourage you to explore all things Zimbra with our team and members of the community.

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 12-05-2006, 06:34 AM
Starter Member
 
Posts: 1
Default Hot backup question on open source?

Hot backup question on open source?
I have been playing with Zimbra for a couple of weeks now and am enjoying it very much. I have a question about hot backups on the opt/zimbra directory.
Tim Brennan mentioned that bad things could happen if the database changes due to an email floating in while a backup is in process. Did he mean it could destroy your mail server or just give you a bad copy to restore from?
I use webmin file system backup for timed daily backups to a USB external just to take off site if needed and I have a RAID mirror for real disaster recovery. The USB drive is just a last resort. Is the hot backup a threat to my stable system? Some input is greatly appreciated.
Reply With Quote
  #2 (permalink)  
Old 08-22-2008, 02:48 AM
Loyal Member
 
Posts: 87
Default

Hi,
Currently Zimbra OSS does not support Hot Backup, according to Zimbra. Zimbra says the database state will not be consistent if you use Hot backup. You can though get some scripts that will automatically shutdown all Zimbra processes and perform the backup. One of such script is on http://swedcore.net. Or rather type backup or script in the forum search.
There a vote going on for Hot backups in Zimbra OSS. Please support.

Cheerio

Last edited by emmaylots; 08-22-2008 at 03:36 AM.. Reason: Correction
Reply With Quote
  #3 (permalink)  
Old 10-05-2008, 04:56 AM
Member
 
Posts: 10
Default HOT Backups in the community edition

I had the problem of backing up (daily) a group of 50 users' mailboxes and taking the server down by doing a cold backup is not really an option. I am using the community version. I am not a Linux guru by any means, however was able to fashion the following scripts that function just fine for my needs, and perhaps can help someone else looking for a similar functionality. I am also up for any "professionalization" that can be done by the real Linux guys seemingly abundant in the Zimbra cosmos.

I am using several steps here. First, a script to export a single user account using zmmailbox. This is used by the next script.
Code:
 #!/bin/bash
 # baccount.sh -- Script to export select Zimbra accounts
 
 USERNAME=$1
 TODAY=`date`
 LOGFILE="/backup/log/backuplog.txt"
 
 echo $TODAY  >> $LOGFILE
 echo $USERNAME >> $LOGFILE
 /opt/zimbra/bin/zmmailbox -z -m $USERNAME gms >> $LOGFILE
 /opt/zimbra/bin/zmmailbox -z -m $USERNAME getRestURL "//?fmt=tgz" > /backup/accounts/$USERNAME.tgz
 echo --------------------- >> $LOGFILE
The above is called using ./baccount.sh username@zimbradomain.com and outputs a file username@zimbradomain.com.tgz to the directory specified.

The script to put this together is as follows.
Code:
 #!/bin/bash
 # get a list of all the accounts and create a script to 
 # execute the baccount.sh script with the email address 
 # of the user as a command line variable
 
 #use zmaccts and extract only the email address from the result
 #then concatenate the "/backup/./baccount.sh <emailaddress>" string for each account and write it to the file 
 
 /opt/zimbra/bin/./zmaccts | grep "@" | awk '{print "/backup/./baccount.sh " $1}' > /backup/accountbackup.sh
 
 # change permissions on the newly created script
 chmod 755 /backup/accountbackup.sh
 
 # Execute the newly created script - this creates a tgz file for each account
 /backup/./accountbackup.sh
  
 # this mounts an SMB drive of a neighboring win machine and copies the backups over - could be anything
 
 mount -t smbfs -o username=username,password=password //windowsMachineName/windowsShare /data
 cp /backup/accounts/*.tgz /data
The above script queries Zimbra for a list of users and generates a shell script called /backup/accountbackup.sh,
changes permissions on the newly created shell script and executes it, calling the baccount.sh script for each email address in the system and subsequently generating a complete mailbox export for each account.

In my example, it then mounts a remote smb volume and copies the backups off-server.

To restore one of these accounts, I use the command:
Code:
 /opt/zimbra/bin/zmmailbox -z -m UserEmailAddress postRestURL "//?fmt=tgz&resolve=reset" /backup/accounts/UserEmailAddress.tgz
I am sure this can all be made more generic and probably simplified, but I have found the flexibility of having these individual mailbox backups to be very very handy when coupled with a weekly full system backup. This procedure is run nightly.
The restore functions best when restored to an empty, newly created account.
Reply With Quote
  #4 (permalink)  
Old 10-05-2008, 10:52 AM
Moderator
 
Posts: 6,236
Default

Just a quick note,
This technique is only possible on 5.0.9+ but there are several reasons to only use it on 5.0.10+:
We found that minor mail store corruptions like a missing blob, size mismatch, or 0 byte db entries can cause a tar operation to halt prematurely/you’ll notice missing data.

You can determine if that is the problem by running “tar tfz account.tgz” and verifying that the listing completes without a corrupt stream error message.

This is similar to bug 31937 (another error handling one was bug 31016) - a fix for truncating or padding mismatched blobs will be included in 5.0.10 NE & we currently plan to release an updated 5.0.10 FOSS as well.

ZCS-to-ZCS Migrations » Zimbra :: Blog
Reply With Quote
  #5 (permalink)  
Old 10-08-2008, 09:38 AM
New Member
 
Posts: 3
Default What would actually break

Hello!

I am adding to this thread as we are discussing this same issue and it's a go/no-go issue for us. Doing cold backup is not really an option, and as it stands now, neither is the commercial version*. So, i have read a little and eveyone basicly says that scary things might happend if i backup a running Zimbra install Scary things is something i genreally wan't to avoid, but it would be nice to now in mor detail what these scary things are.

Say a do an rsync on the entire /opt/zimbra while it's running, and the later on just stop zimbra, rm /opt/zimbra and moving the backup-copy into it's place. I understand that there would be som mails in mysql that is not on disk, and vice versa, but would anything more severe happend (a few mails that "cannot be opened" is pretty acceptable given that the entire RAID would have failed to get me there)? Is it possible that, say, the database would totally corrupted in the backup and the backup totally unusable?

I have tried this operation on a test system under moderate load a few times and it works acceptable every time ....



And please exuse my english


//Joakim


- - -
*(we would only use the backup-feature and paying yearly for 103 accounts (not users, mind you, accounts) just to get a backup feature seems a little har for managament to accept)
Reply With Quote
  #6 (permalink)  
Old 10-08-2008, 09:41 AM
Moderator
 
Posts: 6,236
Default

Welcome to the forums,

That's fine - same technique used here: A_Simple_Shell_Script_Method : Open Source Edition Backup Procedure - Zimbra :: Wiki
Quote:
Live sync before stopping Zimbra to minimize sync time with the services down
Reply With Quote
  #7 (permalink)  
Old 10-08-2008, 09:46 AM
Moderator
 
Posts: 6,236
Default

Wait I re-read that, you're saying you're not going to take a 2nd rsync with Zimbra shutdown; that really is just ripe for a multitude of issues - from PID's to database, index, and store changes. Yes you may get it up and running with slight finesse in a pinch, but I would still take that cold copy occasionally. By doing the 1st rsync you can minimize both your downtime (& risk) considerably.

Plenty of methods:
Backup and Restore Articles - Zimbra :: Wiki
Open Source Edition Backup Procedure - Zimbra :: Wiki

Last edited by mmorse; 10-08-2008 at 09:52 AM.. Reason: links
Reply With Quote
  #8 (permalink)  
Old 10-09-2008, 02:43 AM
New Member
 
Posts: 3
Default

Thanks for that (really quick) reply, active forums are nice.. Yeah, i actually read the script you are referring to (and a few other) and i think it's a really slick solution, two syncs to minimize downtime and let the occational mail just wait on the sending MTA for a few minutes. That downtime is not accepable according to my spec though (say we have a 400k mails, just to STAT those files would take a while no matter how i do it).

Thanks for your answer, i'll have another round thinking about it
Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes


Similar Threads

Why Join?

Registering let's you ask questions, makes it easier to search, displays any files attached to posts, and notifies you about replies.

blog.zimbra.com




 

SEO by vBSEO ©2011, Crawlability, Inc.