Results 1 to 10 of 10

Thread: Daily Full Backup Math Problems

  1. #1
    DougWare is offline Loyal Member
    Join Date
    Dec 2007
    Location
    Raleigh, NC
    Posts
    91
    Rep Power
    7

    Default Daily Full Backup Math Problems

    Can someone please explain to me why 4 days of total backups that are 24G each, take up only 26G on the drive they are backed up to? The backups are stored on the /External-Drive mount.

    See Below:

    HTML Code:
    [root@MailServer1A ~]# df -h
    Filesystem            Size  Used Avail Use% Mounted on
    /dev/mapper/SoftWare_RAID1-rootFS   9.7G  6.8G  2.5G  74% /
    /dev/mapper/SoftWare_RAID1-logFS    992M  137M  804M  15% /var/log
    /dev/md0                            99M   18M   76M   20% /boot
    tmpfs                               486M     0  486M   0% /dev/shm
    216.27.19.206:/External-Drive       426G   26G  401G   7% /External-Drive
    /dev/drbd0                           63G   36G   27G  58% /opt
    Here's the sizes of three of the backup directories...
    HTML Code:
    [root@MailServer1A sessions]# nice du -h --max-depth=1 full-20080225.080012.572
    72K     full-20080225.080012.572/sys
    220K    full-20080225.080012.572/ldap
    24G     full-20080225.080012.572/accounts
    118M    full-20080225.080012.572/shared_blobs
    24G     full-20080225.080012.572
    [root@MailServer1A sessions]# nice du -h --max-depth=1 full-20080224.080009.555/
    72K     full-20080224.080009.555/sys
    220K    full-20080224.080009.555/ldap
    24G     full-20080224.080009.555/accounts
    113M    full-20080224.080009.555/shared_blobs
    24G     full-20080224.080009.555/
    [root@MailServer1A sessions]# nice du -h --max-depth=1 full-20080223.080013.373/
    72K     full-20080223.080013.373/sys
    220K    full-20080223.080013.373/ldap
    24G     full-20080223.080013.373/accounts
    108M    full-20080223.080013.373/shared_blobs
    24G     full-20080223.080013.373/
    Shouldn't they add up to ~72Gs of data?

  2. #2
    Bill Brock is offline Outstanding Member
    Join Date
    May 2007
    Location
    Oklahoma
    Posts
    703
    Rep Power
    9

    Default

    Perhaps backup is using compression?

  3. #3
    ArcaneMagus's Avatar
    ArcaneMagus is offline Moderator
    Join Date
    Feb 2007
    Location
    Portland, OR
    Posts
    1,147
    Rep Power
    10

    Default

    It looks like whatever backup method you are using is creating symbolic links between them...this would explain why they are showing each as ~24.1GB but the total is only 26GB. You might want to do a 'ls -l' within one of the later directories to see whether the majority of the directories/files are just links to other locations.

  4. #4
    DougWare is offline Loyal Member
    Join Date
    Dec 2007
    Location
    Raleigh, NC
    Posts
    91
    Rep Power
    7

    Default

    I'm using Zimbra's backup method, this is the NE version.

    I'll take a look for the links, but I don't understand how it would be smart enough to "link" different versions of backups to each other.

  5. #5
    DougWare is offline Loyal Member
    Join Date
    Dec 2007
    Location
    Raleigh, NC
    Posts
    91
    Rep Power
    7

    Default

    I've looked, and I didn't see any obvious links. (Using ls -l -R |grep "lrw")

    My backup (which should grow to a total of 14 days worth) grew to 27G last night, still pretty small.

    Can someone from Zimbra shed some light on this?

    Doug

  6. #6
    ntharani is offline Partner (VAR/HSP)
    Join Date
    May 2007
    Location
    London
    Posts
    3
    Rep Power
    8

    Default Similar but opposite problem

    I wish I had your problem, or something similar!

    We are using Network Edition:

    We are backing up /opt/zimbra/backup to an external drive. The total size of this directory structure at the moment is 10GB.

    Day to day the entire directory might grow by around 50-100 MB.

    Assuming a standard backup schedule within Zimbra:

    Day 1: Full = ~10 GB
    Day 2: Incremental = +60 MB
    Day 3: Incremental = +60MB
    ...
    Day 7: Full
    Day 8: Incremental
    ...
    Day 13 Incremental
    Day 14 = Day 1 (Start again)

    Now when I mirror this type of setup to an external backup setup, we see the following:

    Day 1: 10GB ... /opt/zimbra/backup
    Day 2: ~10.06GB ... /opt/zimbra/backup
    Day 3: ~10.06GB ... /opt/zimbra/backup
    etc..

    It appears that each day Zimbra does an incremental backup it somehow alters the files completely to a third party system that reviews the directory, so it appears as though there have been 10 GB of changes each day.

    As you can imagine this is quite expensive. Is this a bug?

    Thanks,

  7. #7
    DougWare is offline Loyal Member
    Join Date
    Dec 2007
    Location
    Raleigh, NC
    Posts
    91
    Rep Power
    7

    Default

    Actually, I would expect zimbra to change almost each file. It uses a database, so each time a change (email is sent or arrives), the entire database file changes.

    Differential and Incremental backups are not ideal solutions in this case. You might as well perform full backups. If you ever needed to perform a restore, it would save you the headache of multiple restores (the full and incremental).

    Doug

  8. #8
    uxbod's Avatar
    uxbod is offline Moderator
    Join Date
    Nov 2006
    Location
    UK
    Posts
    8,017
    Rep Power
    24

    Default

    DougWare, if you do a fuser on some of those files do they show a PID attached? I am just wondering whether something is holding the file open, and therefore it is not reporting its true size.

  9. #9
    ntharani is offline Partner (VAR/HSP)
    Join Date
    May 2007
    Location
    London
    Posts
    3
    Rep Power
    8

    Default

    Quote Originally Posted by DougWare View Post
    Actually, I would expect zimbra to change almost each file. It uses a database, so each time a change (email is sent or arrives), the entire database file changes.

    Differential and Incremental backups are not ideal solutions in this case. You might as well perform full backups. If you ever needed to perform a restore, it would save you the headache of multiple restores (the full and incremental).

    Doug
    Hi Doug, Thanks for the reply - but this should not be the case in a backup scenario.

    /opt/zimbra/backup/sessions/

    Assume the following

    /backupfolder_session; Size; Cumulative Change; External Backup Ref

    /day1_full; 10GB; Cumulative change = 10GB; 10GB
    /day2_incr; 100MB; Cumulative change = 10.1; 10GBday1 (no diff) + 100 day2

    maybe my logic is flawed here - but it seems like the zimbra backup does a lookup on the previous days backup to calculate a diff against the current day and in so doing - changes the date/attribute of the previous day so to an external backup the whole thing looks like it has changed.

    Happy to hear your thoughts.

  10. #10
    DougWare is offline Loyal Member
    Join Date
    Dec 2007
    Location
    Raleigh, NC
    Posts
    91
    Rep Power
    7

    Default

    My bad, I misunderstood and thought you were backing up the /opt/zimbra directories.

    From what I can tell, even Zimbra's backup changes almost everything.

    I know this make no sense, but it look like that the last backup in the rotation is a full backup, everything else is only a diff. I could be wrong, either way, it always looks weird to me.

    Doug

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Replies: 658
    Last Post: 04-04-2014, 09:01 AM
  2. Replies: 2
    Last Post: 09-22-2011, 01:44 AM
  3. Restore single account failed from full backup
    By fmodola in forum Administrators
    Replies: 4
    Last Post: 06-20-2007, 05:27 AM
  4. copying only last full backup to store offsite
    By riccadus in forum Administrators
    Replies: 0
    Last Post: 01-11-2007, 01:29 AM
  5. NE Full backup problem
    By Matt in forum Administrators
    Replies: 4
    Last Post: 11-24-2006, 03:29 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •