Results 1 to 10 of 10

Thread: Hot Backup Vs Cold Backup in CE version

  1. #1
    blason is offline Elite Member
    Join Date
    Sep 2011
    Posts
    255
    Rep Power
    3

    Default Hot Backup Vs Cold Backup in CE version

    Hi Folks,

    I have a question regarding OSS version, I took a backup of my zimbra server using rsync while my server is running i.e I didn't stop any of the services and even after that I managed to restore the system successfully so my question is do I really need to stop the services while taking backup? Will this cause any consequences rather than doing a cold backup?

  2. #2
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,480
    Rep Power
    56

    Default

    Quote Originally Posted by blason View Post
    Hi Folks,

    I have a question regarding OSS version, I took a backup of my zimbra server using rsync while my server is running i.e I didn't stop any of the services and even after that I managed to restore the system successfully so my question is do I really need to stop the services while taking backup? Will this cause any consequences rather than doing a cold backup?
    Yes you do need to stop the services. You may get lucky and successfully restore a backup (you may have problems later on) but the files cannot be guaranteed to be in a consistent state when you restore them. If this is a production mail server the question I'd ask: are you prepared to lose all your mail with a failed restore? If the answer to that is "yes" then carry on without stopping the services, if the answer is "no" the...... you can see where I'm going with this, I guess.
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  3. #3
    blason is offline Elite Member
    Join Date
    Sep 2011
    Posts
    255
    Rep Power
    3

    Default

    Well, thanks Phoenix but honestly I was not lucky for once or twice but I did each and every backup testing on my test mail server for about 10 to 15 times and I was successfully restored that every time.
    And trust me that actually forced me to drop a question in forum to understand others voices. I agree that services should be stopped to clean pickup the files but that never happened with me;so wondering why cant hot backup be done with OSS version.

    You wouldn't believe even I tried with rsync first while services are running then stopped the services and initiated backup again just to test the setup and I was again able to restore the entire system cleanly.

  4. #4
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,480
    Rep Power
    56

    Default

    Quote Originally Posted by blason View Post
    I agree that services should be stopped to clean pickup the files but that never happened with me;so wondering why cant hot backup be done with OSS version.
    I've already answered this, the files cannot be guaranteed to be in a consistent state. The fact that it has 'never' happened to you does not mean that it won't ever happen to you.

    My original comment still stands, if you're happy to do that and you are also happy that one day you may need to restore the server and it won't work and you lose all your email then carry on doing what you're doing. It's totally your choice, I'm not here to convince you otherwise - you asked a question and you've had my answer. If you need further confirmation then search the forums, there are several threads from various Zimbra employees (notably the director of engineering (I think that's his title) stating the same thing.

    Quote Originally Posted by blason View Post
    You wouldn't believe even I tried with rsync first while services are running then stopped the services and initiated backup again just to test the setup and I was again able to restore the entire system cleanly.
    Why wouldn't I believe it? You've said that's the case and I believe you, however, you seem not to believe my reply on why it's a bad idea.
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  5. #5
    Tripple is offline Elite Member
    Join Date
    Apr 2009
    Posts
    304
    Rep Power
    6

    Default

    And what's the load on your testserver while you're taking a hot backup?
    I'm using a script found on this site who's taking a hot backup, shut down Zimbra and take a cold backup only to keep the downtime to a strict minimum.

  6. #6
    blason is offline Elite Member
    Join Date
    Sep 2011
    Posts
    255
    Rep Power
    3

    Default

    Nothing much hardly 30-35 users created with arnd 1.5 Gb of individual user's pst pumped in.

    I know and I am following the same stuff with rsync the entire directory then shutdown the services and rsync it again.

    BTW just wondering have you guys or any one able to figure it out as what NE version does to take a HOT backup that CE can not do it? I mean just out of curiosity.

  7. #7
    bofh is offline Elite Member
    Join Date
    May 2010
    Posts
    272
    Rep Power
    5

    Default

    Hello Blason

    First of all its a premier - i agree with Phoenix - first time in my life so if oyu dont believe him believe me.

    So you cannot say your system is restored cleanly - you simply cant - you will see if i dont if an error occurs but theres no real proof that it did

    even if you check mailbox integrity and stuff like that this isnt a garantuee or any kind of proof everything is thing

    you must understand zimbra uses -
    a dynamic store on filebasis - so every file in there is sorted by IDs
    those IDs are stored in the mysql backend - so the relation mysql and filesystem is very important
    and to make things worse you got also an ldap store for all user account information things

    in addition you cannot backup mysql while its running so easy
    there are already aproches todo so but its way more comlicated - you have to store transaction logs and the bin position and restore them correctly - but these are specialsed products and theres a reason why almost every major backupsoftware on linux implemented a dump by mysql dump instead of simple filecopy

    SO NO hotbackup of mysql - always a bad idea
    hot backup mysql on zimbra because of massive write operations - desaster waiting to happen

    Even if your hotbackp on mysql - and on ldap works - imagine the follwoing -

    your rsync backuped mysql - fisnish
    now a mail comes ins - zimbra do its magic wirtie to filesystem and mysql
    now you backup the filesystem

    when you restore you restore only the file but not the mysql data (or vice versa) -
    ife no idea what will happen if zimbra decides to use that "unused" id for another mail again or - maybe worse you got a mysql db entry but no data on the restore ....

    (i know ife simpliefed things...)




    So defently hotbackup for restore a nogo -

    however you can use hotbackup to reduce the downtime

    you simply make several hotbackups a day and later on a certain time one coldbackup - your downtime reduce dramatically - down to a few minutes -

    another adavantage is thats usually not so bad to restart the java vm once in a while java likes reboots hehe



    in order to use my hotbackup/coldbackup combination you must be able todo incremental backups and be able to restore a certain version (exactly that version where you made the coldbackup)

    ther a lot of rsync script they can do that easy

    i dont use rsync i use another incremental backup but i can share you my prebackup and postbackup scripts - because i use the same job with a different time squedule i decide by time on which shedule im doing the shutdown

    Prebackup
    Code:
    #!/bin/bash
    
    
    chour=$(date +%H)
    #declare -i chour
    
    if ([ "$chour" -ge 5 ] && [ "$chour" -lt 6 ])  ; then
             echo "Its" $chour "- Shutdown"
             sudo -H -u zimbra /opt/zimbra/bin/zmcontrol stop
    else
    echo "Its" $chour "- No shutdown"
    fi

    Postbackup
    Code:
    #!/bin/bash
    
    
    chour=$(date +%H)
    #declare -i chour
    
    if ([ "$chour" -ge 5 ] && [ "$chour" -lt 11 ])  ; then
             echo "Its" $chour "- Startup"
            sudo -H -u zimbra /opt/zimbra/bin/zmcontrol start
    else
    
    echo "Its" $chour "- No Startup"
    fi

    They are very simple - prebackup decide when to shutdown
    postbackup restarts at the same time

    i dont wanted to make a start zimbra all the time if its not nessesary even if it should do no damage - i simply dont do such things

    so just define your incremental rsync job always with those script pre/post
    and set the script on the time of rsync backup you wanna do cold

  8. #8
    bofh is offline Elite Member
    Join Date
    May 2010
    Posts
    272
    Rep Power
    5

    Default

    PS: to your question about zm NE backup
    Its quiet simple they use their transaction logs
    they dump ldap and mysql and stuff like that - so they dont simply copy files

    also it wasnt even easy for them to find the right way - backup had issues till this year with some major bugfixes maybe still has - its not a complain - just to show that it isnt so simple even for people how developed that storing system

    pps: the biggest part of my 3minutes 25 sec downtime is that my backupsoftware read out its database and compare changes to every file
    that - nad startup and shutdown are the 2 major timekiller - the amount of data with 5 hotbackups during the day (3 alone between 1am and 5:30 )
    is almost nothing a few hundred megs - so downtime isnt an issue

  9. #9
    blason is offline Elite Member
    Join Date
    Sep 2011
    Posts
    255
    Rep Power
    3

    Default

    Thanks bofh for such an extensive explanation I must appreciate that. instead I would recommend you to write and Wiki about the background stuff happens while backup in progress or backup 101 something like that and definitely its not about disbelief on anybody's comment its about the fact that I experienced though without knowing all this background hence wanted to know what exactly happens when we initiate backup.

    And regarding NE I was damn sure its just not copying the stuff, it wouldn't be, again wanted to know what exactly happens behind the curtain in NE as well.

  10. #10
    bofh is offline Elite Member
    Join Date
    May 2010
    Posts
    272
    Rep Power
    5

    Default

    i addmit iam lazy - and i hate wiki writing - feel free to take any of my stuff and post it under your name its for everyone

    btw: im aware of that rsync script which uses also hot prebackup and cold backup afterwards

    but i strongly recommend todo with scripts i wrote
    for a simple reason - you dont have control when coldbackup runs if you got a spike of data or downtime on the backuppath/backupline

    means lets say you use that script its starts at 2am - works usually 1 hour then coldbackup 10 minutes - regular downtime 3am till :10

    but - if you got a lot of data or you line for backup slows down - your coldbackup might start at 10am - problem

    if the line breaks (i cant say that for shure but it think it will be that way) the hotprebackup fails and cold starts
    so you have to replicate all the data while its down - worst case couple hours downtime


    thats why i work with those scripts - i will only go down at 5:30
    and backup job runtime is set on a max of 30 minutes - so worst case 31min downtime if something bad happens bevore

    worst case 1 day an incomplete backup (and i recive a warning) but not a downtime of xxx hours

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Replies: 658
    Last Post: 04-04-2014, 09:01 AM
  2. Hot backup question on open source?
    By cfigurelli in forum Administrators
    Replies: 7
    Last Post: 10-09-2008, 01:43 AM
  3. Debian - Upgrade from 4.5.6
    By magikman in forum Installation
    Replies: 1
    Last Post: 01-19-2008, 12:59 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •