Results 1 to 8 of 8

Thread: Hot backup question on open source?

  1. #1
    cfigurelli is offline Starter Member
    Join Date
    Nov 2006
    Posts
    1
    Rep Power
    8

    Default Hot backup question on open source?

    Hot backup question on open source?
    I have been playing with Zimbra for a couple of weeks now and am enjoying it very much. I have a question about hot backups on the opt/zimbra directory.
    Tim Brennan mentioned that bad things could happen if the database changes due to an email floating in while a backup is in process. Did he mean it could destroy your mail server or just give you a bad copy to restore from?
    I use webmin file system backup for timed daily backups to a USB external just to take off site if needed and I have a RAID mirror for real disaster recovery. The USB drive is just a last resort. Is the hot backup a threat to my stable system? Some input is greatly appreciated.

  2. #2
    emmaylots is offline Loyal Member
    Join Date
    May 2008
    Location
    Nigeria
    Posts
    92
    Rep Power
    6

    Default

    Hi,
    Currently Zimbra OSS does not support Hot Backup, according to Zimbra. Zimbra says the database state will not be consistent if you use Hot backup. You can though get some scripts that will automatically shutdown all Zimbra processes and perform the backup. One of such script is on http://swedcore.net. Or rather type backup or script in the forum search.
    There a vote going on for Hot backups in Zimbra OSS. Please support.

    Cheerio
    Last edited by emmaylots; 08-22-2008 at 02:36 AM. Reason: Correction

  3. #3
    SGIA's Avatar
    SGIA is offline Member
    Join Date
    Sep 2008
    Location
    Germany
    Posts
    13
    Rep Power
    6

    Default HOT Backups in the community edition

    I had the problem of backing up (daily) a group of 50 users' mailboxes and taking the server down by doing a cold backup is not really an option. I am using the community version. I am not a Linux guru by any means, however was able to fashion the following scripts that function just fine for my needs, and perhaps can help someone else looking for a similar functionality. I am also up for any "professionalization" that can be done by the real Linux guys seemingly abundant in the Zimbra cosmos.

    I am using several steps here. First, a script to export a single user account using zmmailbox. This is used by the next script.
    Code:
     #!/bin/bash
     # baccount.sh -- Script to export select Zimbra accounts
     
     USERNAME=$1
     TODAY=`date`
     LOGFILE="/backup/log/backuplog.txt"
     
     echo $TODAY  >> $LOGFILE
     echo $USERNAME >> $LOGFILE
     /opt/zimbra/bin/zmmailbox -z -m $USERNAME gms >> $LOGFILE
     /opt/zimbra/bin/zmmailbox -z -m $USERNAME getRestURL "//?fmt=tgz" > /backup/accounts/$USERNAME.tgz
     echo --------------------- >> $LOGFILE
    The above is called using ./baccount.sh username@zimbradomain.com and outputs a file username@zimbradomain.com.tgz to the directory specified.

    The script to put this together is as follows.
    Code:
     #!/bin/bash
     # get a list of all the accounts and create a script to 
     # execute the baccount.sh script with the email address 
     # of the user as a command line variable
     
     #use zmaccts and extract only the email address from the result
     #then concatenate the "/backup/./baccount.sh <emailaddress>" string for each account and write it to the file 
     
     /opt/zimbra/bin/./zmaccts | grep "@" | awk '{print "/backup/./baccount.sh " $1}' > /backup/accountbackup.sh
     
     # change permissions on the newly created script
     chmod 755 /backup/accountbackup.sh
     
     # Execute the newly created script - this creates a tgz file for each account
     /backup/./accountbackup.sh
      
     # this mounts an SMB drive of a neighboring win machine and copies the backups over - could be anything
     
     mount -t smbfs -o username=username,password=password //windowsMachineName/windowsShare /data
     cp /backup/accounts/*.tgz /data
    The above script queries Zimbra for a list of users and generates a shell script called /backup/accountbackup.sh,
    changes permissions on the newly created shell script and executes it, calling the baccount.sh script for each email address in the system and subsequently generating a complete mailbox export for each account.

    In my example, it then mounts a remote smb volume and copies the backups off-server.

    To restore one of these accounts, I use the command:
    Code:
     /opt/zimbra/bin/zmmailbox -z -m UserEmailAddress postRestURL "//?fmt=tgz&resolve=reset" /backup/accounts/UserEmailAddress.tgz
    I am sure this can all be made more generic and probably simplified, but I have found the flexibility of having these individual mailbox backups to be very very handy when coupled with a weekly full system backup. This procedure is run nightly.
    The restore functions best when restored to an empty, newly created account.

  4. #4
    mmorse's Avatar
    mmorse is offline Moderator
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    20

    Default

    Just a quick note,
    This technique is only possible on 5.0.9+ but there are several reasons to only use it on 5.0.10+:
    We found that minor mail store corruptions like a missing blob, size mismatch, or 0 byte db entries can cause a tar operation to halt prematurely/you’ll notice missing data.

    You can determine if that is the problem by running “tar tfz account.tgz” and verifying that the listing completes without a corrupt stream error message.

    This is similar to bug 31937 (another error handling one was bug 31016) - a fix for truncating or padding mismatched blobs will be included in 5.0.10 NE & we currently plan to release an updated 5.0.10 FOSS as well.

    ZCS-to-ZCS Migrations &#187; Zimbra :: Blog

  5. #5
    jalakas is offline New Member
    Join Date
    Oct 2008
    Location
    Sweden
    Posts
    3
    Rep Power
    6

    Default What would actually break

    Hello!

    I am adding to this thread as we are discussing this same issue and it's a go/no-go issue for us. Doing cold backup is not really an option, and as it stands now, neither is the commercial version*. So, i have read a little and eveyone basicly says that scary things might happend if i backup a running Zimbra install Scary things is something i genreally wan't to avoid, but it would be nice to now in mor detail what these scary things are.

    Say a do an rsync on the entire /opt/zimbra while it's running, and the later on just stop zimbra, rm /opt/zimbra and moving the backup-copy into it's place. I understand that there would be som mails in mysql that is not on disk, and vice versa, but would anything more severe happend (a few mails that "cannot be opened" is pretty acceptable given that the entire RAID would have failed to get me there)? Is it possible that, say, the database would totally corrupted in the backup and the backup totally unusable?

    I have tried this operation on a test system under moderate load a few times and it works acceptable every time ....



    And please exuse my english


    //Joakim


    - - -
    *(we would only use the backup-feature and paying yearly for 103 accounts (not users, mind you, accounts) just to get a backup feature seems a little har for managament to accept)

  6. #6
    mmorse's Avatar
    mmorse is offline Moderator
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    20

    Default

    Welcome to the forums,

    That's fine - same technique used here: A_Simple_Shell_Script_Method : Open Source Edition Backup Procedure - Zimbra :: Wiki
    Live sync before stopping Zimbra to minimize sync time with the services down

  7. #7
    mmorse's Avatar
    mmorse is offline Moderator
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    20

    Default

    Wait I re-read that, you're saying you're not going to take a 2nd rsync with Zimbra shutdown; that really is just ripe for a multitude of issues - from PID's to database, index, and store changes. Yes you may get it up and running with slight finesse in a pinch, but I would still take that cold copy occasionally. By doing the 1st rsync you can minimize both your downtime (& risk) considerably.

    Plenty of methods:
    Backup and Restore Articles - Zimbra :: Wiki
    Open Source Edition Backup Procedure - Zimbra :: Wiki
    Last edited by mmorse; 10-08-2008 at 08:52 AM. Reason: links

  8. #8
    jalakas is offline New Member
    Join Date
    Oct 2008
    Location
    Sweden
    Posts
    3
    Rep Power
    6

    Default

    Thanks for that (really quick) reply, active forums are nice.. Yeah, i actually read the script you are referring to (and a few other) and i think it's a really slick solution, two syncs to minimize downtime and let the occational mail just wait on the sending MTA for a few minutes. That downtime is not accepable according to my spec though (say we have a 400k mails, just to STAT those files would take a while no matter how i do it).

    Thanks for your answer, i'll have another round thinking about it

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Installation Problem
    By Ju-Pao in forum Installation Help
    Replies: 39
    Last Post: 05-15-2007, 02:54 PM
  2. Error message in Server status
    By Max Ma in forum Installation
    Replies: 20
    Last Post: 04-19-2007, 08:55 AM
  3. Basic Open Source Question in contact of Zimbra
    By BoomerangThree in forum Developers
    Replies: 4
    Last Post: 10-15-2006, 07:16 AM
  4. Open Source Not Restarting From Backup Correctly
    By alanj_29672 in forum Administrators
    Replies: 2
    Last Post: 09-18-2006, 06:00 AM
  5. M3 problem with shares
    By titangears in forum Users
    Replies: 4
    Last Post: 01-12-2006, 01:01 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •