Page 1 of 2 12 LastLast
Results 1 to 10 of 11

Thread: Zimbra Open Source Backup without stopping the server

  1. #1
    Join Date
    Nov 2009
    Posts
    11
    Rep Power
    5

    Default Zimbra Open Source Backup without stopping the server

    Hi!

    This week I tried to backup my zimbra server to prepare me for a crash situation.
    I'm using ubuntu 8.04 and the zimbra version is the latest freeware.

    steps
    1- I did an image with clonezila (after creating the users)
    2- Send some emails, create some contacts and some meetings (just to test)
    3- Backup the folder /opt/zimbra/store and the mysql database with the software Mysql Administrator
    4- "Crash it" (or simply restore you harddrive with clonezila's image)
    5- After restoring, delete the folder /opt/zimbra/store and replace it with your backup that should be daily (or whatever you want)
    6- Restore the mysql database using the software Mysql Administrator (this must be after the step 5)

    Ok, so, I know that the free version should not be backed up without stopping the server and I should not touch the database. But in fact this works...Did I have luck or this is suppose to work??

    Comment please

  2. #2
    bdial's Avatar
    bdial is offline Moderator
    Join Date
    Jul 2007
    Location
    Baltimore
    Posts
    1,649
    Rep Power
    10

    Default

    when you do step 3 what is your timing like? like do you backup /opt/zimbra/store and then backup the mysql db? How would you handle the instance where after you've backed up a particular user's store foler they receive an e-mail before the db backup runs. In that case the db would have record of the e-mail but your store backup wouldn't have the e-mail

  3. #3
    Join Date
    Nov 2009
    Posts
    11
    Rep Power
    5

    Default

    Quote Originally Posted by bdial View Post
    when you do step 3 what is your timing like? like do you backup /opt/zimbra/store and then backup the mysql db? How would you handle the instance where after you've backed up a particular user's store foler they receive an e-mail before the db backup runs. In that case the db would have record of the e-mail but your store backup wouldn't have the e-mail
    That true..I run both at 2am and they are fast (database 30 sec and store 1-2 min)
    I understand your point but if that is the only reason why the restore might fail I think that is a good risk to take. And my company doesn't receive many email and barely never at night. So for us is a good solution.

    But if there is any other reason that can make the restoring fail, please tell me.

    thanks

  4. #4
    bdial's Avatar
    bdial is offline Moderator
    Join Date
    Jul 2007
    Location
    Baltimore
    Posts
    1,649
    Rep Power
    10

    Default

    ok well it sounds like you know what you're getting into.

    The 2 things you need to worry about when backing up filesystem only are the mysql db and lucene index. I don't know how mysql administrator works but it probably dumps the db to flatfiles. This is pretty much how zimbra's network edition backup does it too and this is a good way to do things.

    I'm not sure if you can do the same kind of thing with the lucene index which is pretty much a db as well. Of course you can always reindex any accounts that have corrupted indexes.

    The issues surrounding backing up filesystems with live databases is the event where you copy a db that is in the middle of a transaction. This is why zimbra puts accounts into maintenance mode while it backs up indexes. It sounds like this might nto happen often for you if you don't have much e-mail coming in while your backup is running.

    Make sure you're backing up your ldap db as well if you're trying to do a full server disaster recovery backup

  5. #5
    Join Date
    Nov 2009
    Posts
    11
    Rep Power
    5

    Default

    Quote Originally Posted by bdial View Post
    ok well it sounds like you know what you're getting into.

    The 2 things you need to worry about when backing up filesystem only are the mysql db and lucene index. I don't know how mysql administrator works but it probably dumps the db to flatfiles. This is pretty much how zimbra's network edition backup does it too and this is a good way to do things.

    I'm not sure if you can do the same kind of thing with the lucene index which is pretty much a db as well. Of course you can always reindex any accounts that have corrupted indexes.

    The issues surrounding backing up filesystems with live databases is the event where you copy a db that is in the middle of a transaction. This is why zimbra puts accounts into maintenance mode while it backs up indexes. It sounds like this might nto happen often for you if you don't have much e-mail coming in while your backup is running.

    Make sure you're backing up your ldap db as well if you're trying to do a full server disaster recovery backup
    well..Now I'm a little worried. Where is the ldap db and how often does it change?
    And can you tell me a little more about the lucene indexes in zimbra because I didn't find anything about it at the wiki.

    Thanks a lot for your time

  6. #6
    zmailcc is offline Member
    Join Date
    Jan 2007
    Location
    Delaware, USA
    Posts
    12
    Rep Power
    8

    Default

    What is the reason for wanting to not stop the server? With the open source version, there is always a finite probability that a message will arrive during the backup of a live server and result in databases and files being out of sync; making a reliable restore point questionable.

    The safest and easiest way to backup the open source version is to use one of several backup scripts found in the wiki or these forums. I found the LVM snapshot approach to be the best for our servers. I modified a script to fit our needs. It only requires the servers to be offline for about 80 seconds to stop the server, create an LVM snapshot, and restart the server. Most of that time is in stopping and starting Zimbra, the snapshot only takes a second. The snapshot can then be saved using tar or rsync and it is a perfect copy of the static /opt/zimbra directory. The actual backup of the snapshot to a tar.gz file takes about 2.5 hours on our server. The backup is done at 1:00AM and goes completely unnoticed. I have tested the restore operation several times on a backup server and once needed to use it to recover from a failed DRAM in our primary server -- it worked every time.

    If you do not have /opt/zimbra on a logical volume, the rsync approach works good. There are some good scripts in the wiki for doing this. They first do an rsync of /opt/zimbra, then stop zimbra, do a second rsync to get what changed, and restart zimbra. On our server the second rsync on the stopped server takes about 5 to 10 minutes, which shows how much can change on a running server.

  7. #7
    raj's Avatar
    raj
    raj is offline Moderator
    Join Date
    Oct 2005
    Location
    USA, Canada and India
    Posts
    777
    Rep Power
    10

    Default

    Quote Originally Posted by zmailcc View Post
    On our server the second rsync on the stopped server takes about 5 to 10 minutes, which shows how much can change on a running server.
    What you do is you do rsync twice or 3 times before stopping and you will see this 5 min will be < 2 min or less

    for us at 2 am everyone is sleeeeeeeeeping even on a busy server ..so 2 min is no big deal ..better than no backup.

    Raj
    i2k2 Networks
    Dedicated & Shared Zimbra Hosting Provider

  8. #8
    bdial's Avatar
    bdial is offline Moderator
    Join Date
    Jul 2007
    Location
    Baltimore
    Posts
    1,649
    Rep Power
    10

    Default

    Quote Originally Posted by ruben.mourinho View Post
    well..Now I'm a little worried. Where is the ldap db and how often does it change?
    And can you tell me a little more about the lucene indexes in zimbra because I didn't find anything about it at the wiki.

    Thanks a lot for your time
    The ldap data can be dumped easily by the command

    Code:
    /opt/zimbra/libexec/zmslapcat /path/to/ldap/backup/dir
    that will dump the ldap db to a flat file just like you do with mysql.

    The indexes are held in /opt/zimbra/index. I'm not sure where or if theres more documentation on them.

  9. #9
    zmailcc is offline Member
    Join Date
    Jan 2007
    Location
    Delaware, USA
    Posts
    12
    Rep Power
    8

    Default

    Quote Originally Posted by raj View Post
    What you do is you do rsync twice or 3 times before stopping and you will see this 5 min will be < 2 min or less

    for us at 2 am everyone is sleeeeeeeeeping even on a busy server ..so 2 min is no big deal ..better than no backup.

    Raj
    Thanks for the tip Raj. That makes the rsync approach very attractive.

    I have used the LVM snapshot backup approach since I setup my first Zimbra server using Zimbra 4.0. The forums and wiki really helped -- many of the posts were yours. I use an rsync based script to make a backup just before applying upgrades and will add the second live rsync to it.

    I mainly wanted to show that a lot can change on a live server while files and databases are being copied to a backup; making it risky to make a backup of the open source edition without stopping the server. I imagine the Network Edition has extra logic to keep database and message stores in sync during a live backup.

  10. #10
    Mistoffeles is offline Senior Member
    Join Date
    Oct 2007
    Posts
    70
    Rep Power
    7

    Default

    Quote Originally Posted by ruben.mourinho View Post
    well..Now I'm a little worried. Where is the ldap db and how often does it change?
    As far as I know LDAP data only changes when you make changes to the user list, such as adding/editing/deleting user accounts or adding/editing/deleting domains. This shouldn't happen too much if you just have one domain and all edits to the addressbook are done by administrators.
    - Misty

Page 1 of 2 12 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] Zimbra logwatch.
    By nishith in forum Administrators
    Replies: 5
    Last Post: 06-10-2009, 04:42 PM
  2. Replies: 8
    Last Post: 01-20-2009, 01:06 PM
  3. Zimbra spam system
    By rajahd in forum Administrators
    Replies: 9
    Last Post: 04-16-2008, 07:25 PM
  4. Can't start Zimbra!
    By zibra in forum Administrators
    Replies: 5
    Last Post: 03-22-2007, 11:34 AM
  5. dspam logrotate errors
    By michaeln in forum Users
    Replies: 7
    Last Post: 02-19-2007, 12:45 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •