Backup Questions

[Chewie71]

I'm just curious how other larger sites are handling their Zimbra backups.

We have 5 mailbox servers, each with 30-40GB of /opt/zimbra/store data and another 500-600GB of /opt/zimbra/hsm data. Our /opt/zimbra/backup volumes are 1TB.

We currently run our backups with --noZip because it improves the speed of the backup over running it zipped. We also have auto-grouped turned on and set to 7 so if I understand correctly every account gets a full at least once every 7 days....and this spreads our fulls out over multiple days. I don't know if auto-grouped is working though as the zmbackupquery shows me this...

18th full: 14 hours
17th incr: 45 mins
16th incr: 45 mins
15th incr: 45 mins
14th incr: 45 mins
13th incr: 45 mins
12th incr: 70 mins
11th full: 13.5 hours
...etc

I would think if auto-grouped was working the backup length would be nearly the same every day.

The other problem we're having is storage. We can't keep more than maybe 20 days of backups online on a couple of our mailbox servers because we run out of space. We do have more space we can give to these backup volumes, but I'm wondering if we should be getting more efficient use out of what we already have.

I suppose we could start zipping backups again but then the backup jobs take even longer to complete, at least from my experience. We could auto-group to ease the impact of any single full backup on a given day but it doesn't seem as if my auto-grouped settings are working.

Anyway....just trying to see what others are doing and see what the "best practices" are for sites with LOTS of mail data where fulls take a LONG time to complete.

Thanks,
Matt

[Chewie71]

Ah....found this and I need to run the "zmschedulebackup -D" command to get my auto-group settings to take effect.
Ajcody-Backup-Restore-Issues - Zimbra :: Wiki

So I guess I can fix that. But I'm still curious what others consider the most efficient (storage and time wise) method for backing up big mailstores.

Thanks,
Matt

[ewilen]

Just to be clear, there are two kinds of "zipped" backups:

--zipStore (default for 6.0) stores everything in zipped files without compression. My experience with this is: it's slow, and it uses a lot more space than --noZip, because it can't use hardlinks between files that are common to consecutive full backups.

--zip (option for 6.0) stores everything in zipped files with compression. I'd expect that this will save quite a bit of space, but it will be even slower than --zipStore.

Beyond that, I don't have any suggestions except possibly archiving your backups to tape after they're more than X days old. However if you use --noZip, you should make sure that your tape backup tool supports hardlinks, or you'll waste a lot of time/space copying duplicate files.

[Hivos]

In my experience using the --noZip option, is the fastest way to backup and uses the lowest amount of diskspace. It's a bit of a mystery to me why this is not the default behaviour.

Anyway, we have around 250GB of data and keep 30 days of backups, which takes up ~300GB (Yes, using --noZip). Datastore is on 15k SAS disks in RAID10, whereas the backup is located on 10k SAS disks in RAID 0. While the backup volume is not redundant at least it's fast, which is important for a quick restore / disaster-recovery (I've experienced waiting 17 hours for a disaster recovery to complete from slow disks).

We have a Disk-to-Disk-to-Tape procedure in place: the /opt/zimbra/backup folder is rsynced daily to a backup-server, which does an old-school tar to tape (tar rules!). Both rsync and tar support hardlinks by the way. Also don't forget to backup your /opt/zimbra/conf and /opt/zimbra/redolog directories, which hold your config and "redologs" (all email that has not been backed up yet!). In the disaster recovery mentioned above we lost "only" 15 minutes of email, instead of a whole day, because we had the redologs.