Community Edt (6.0.6) Smart Backp?

[mhakali]

Hello.

I have been reading the wiki articles regarding backup & restore. I've been searching the forums and looked through user published scripts in regards to backups.

All backup strategies inclusive the ones in the wiki basically says; "pack up /opt/zimbra, move it somewhere, in case you need a restore, unpack it". This works. But this is not a smart backup strategy.

What files under /opt/zimbra can I exclude?

Do I need to take backup of binaries which exist within the .deb/.rpm files and will be restored by a clean install before unpacking my config anyhow?

Is there some "minimal" list of files that can be backed up in order to perform a full backup of my contents?

And also, while migrating this installation inbetween versions. Upgrading one to the next. I end up with a file tree looking like:

root@imix:/opt/zimbra# ls -1
[...]
amavisd
amavisd-new-2.4.3
amavisd-new-2.5.2
amavisd-new-2.6.4
[...]
clamav
clamav-0.90.2
clamav-0.91.2
clamav-0.92
clamav-0.95.3
[...]
cyrus-sasl
cyrus-sasl-2.1.21.ZIMBRA
cyrus-sasl-2.1.22.3
cyrus-sasl-2.1.22.3z
cyrus-sasl-2.1.23.3z
[...]
java
jdk1.5.0_08
jdk1.5.0_15
jdk1.5.0_17
jdk1.6.0_02
jdk1.6.0_04
jdk1.6.0_16
jetty
jetty-6.1.15
jetty-6.1.22.z6
jetty-6.1.5
[...]
postfix
postfix-2.2.9
postfix-2.4.3.3
postfix-2.4.3.3z
postfix-2.4.3.4z
postfix-2.4.7.5z
postfix-2.6.2.2z
postfix-2.6.5.2z

Can these earlier versions be cleaned up? Why doesn't the install script do so? Is there any risk associated?

Mikael

[LaFong]

All backup strategies inclusive the ones in the wiki basically says; "pack up /opt/zimbra, move it somewhere, in case you need a restore, unpack it". This works. But this is not a smart backup strategy.

Sure it is, depending on how you do it. Using a backup tool, e.g. rsync, dar, which does incrementals or deltas, most backups don't take that long. And most of that time is needed for the store and database.

What files under /opt/zimbra can I exclude?

Excluding none would be best.

Do I need to take backup of binaries which exist within the .deb/.rpm files and will be restored by a clean install before unpacking my config anyhow?

Most people would not do a clean install for a restore. Just restore the installation directory, e.g. /opt/zimbra, and you're done. Even if you had to do a clean install, backing up the entire directory insures not missing anything.

Is there some "minimal" list of files that can be backed up in order to perform a full backup of my contents?

You will need to back up the conf directory, store, and the database, at a minimum. These comprise by far the most files and most size of the typical Zimbra installation. It's not worth dissecting the rest of the installation directory to minimize further what you back up, IMHO.

Can these earlier versions be cleaned up? Why doesn't the install script do so? Is there any risk associated?

It would be nice if upgrades did better cleanup, or at least some procedure in the wiki. In general, you can delete old folder versions, which are not pointed at by a soft link. There may be exceptions however. Absent guidance from Zimbra, you could just move an old directory and see if anything broke.
Some earlier threads on the topic:
[SOLVED] Cleaning up installation folder, delete old files
Cleaning up the Zimbra area

[mhakali]

Originally Posted by LaFong

Sure it is, depending on how you do it. Using a backup tool, e.g. rsync, dar, which does incrementals or deltas, most backups don't take that long. And most of that time is needed for the store and database.

The downside with the incremental backups is that you require to have the other increments taken before that one. And I don't see the justification for snapshotting the binaries just because it doesn't take long or much space

To motivate my question a bit better: All scenarios where I had to perform a restore / disaster recovery included reinstalling a server or atleast Zimbra. It's even in the official backup wiki to install Zimbra with same settings then restore the backup.

Originally Posted by LaFong

Most people would not do a clean install for a restore. Just restore the installation directory, e.g. /opt/zimbra, and you're done. Even if you had to do a clean install, backing up the entire directory insures not missing anything.

Don't you want your package manager (apt/rpm/yum/etc) to be aware that you have installed packages and what files there packages own? If you just install a server, unpack your latest snapshot & start zimbra it will probably work, but it will Not be pretty package management wise.

Originally Posted by LaFong

You will need to back up the store and the database, at a minimum. These comprise by far the most files and most size of the typical Zimbra installation. It's not worth dissecting the rest of the installation directory to minimize further what you back up, IMHO.

Considering that all the collective .deb files are ~425MB I would say this to be considerable backing up a fully deployed server.

Anyhow. Store & database. Directories? Presume that "database" includes both LDAP and MySQL? Presuming config files, ssl certs, etc would be required aswell.

This is the main question in this thread would be if anyone do have an exact list on what's required and used to get your zimbra working doing a zimbra install then restore of files :-)

Originally Posted by LaFong

It would be nice if upgrades did better cleanup, or at least some procedure in the wiki. In general, you can delete old folder versions, which are not pointed at by a soft link. There may be exceptions however. Absent guidance from Zimbra, you could just move an old directory and see if anything broke.
Some earlier threads on the topic:
[SOLVED] Cleaning up installation folder, delete old files
Cleaning up the Zimbra area

Excellent tip! Thank you.

Mikael

[LaFong]

Originally Posted by mhakali

The downside with the incremental backups is that you require to have the other increments taken before that one. And I don't see the justification for snapshotting the binaries just because it doesn't take long or much space

So, you want to take a full backup every time, but just of the minimum files? The average backup time will be considerably longer, and take up more room. For instance, a full backup of our server would take 70GB or so, just for the minimum files. The other files take up much less than 1GB. We do a full once a week, but the diffs during the week only take several hundred MB.

To motivate my question a bit better: All scenarios where I had to perform a restore / disaster recovery included reinstalling a server or atleast Zimbra. It's even in the official backup wiki to install Zimbra with same settings then restore the backup.

Our backup software can do a bare metal restore, though I would end up copying over the /opt/zimbra backup because the dbs and un-synced message files are backed up with Zimbra stopped to insure consistency.

Don't you want your package manager (apt/rpm/yum/etc) to be aware that you have installed packages and what files there packages own? If you just install a server, unpack your latest snapshot & start zimbra it will probably work, but it will Not be pretty package management wise.

Only if you have to restore the entire server. If, instead, you have db corruption, you don't have to restore the whole server, just /opt/zimbra. If you did have to restore the whole server, you would reinstall Zimbra, as you've noted. Then, just copy over /opt/zimbra, done.

Considering that all the collective .deb files are ~425MB I would say this to be considerable backing up a fully deployed server.

The .deb files are not in /opt/zimbra, are they? I'm on CentOS, and the rpm installer is elsewhere. You don't need to back up the installer, but the binaries in /opt/zimbra are a very small portion of its overall size.

Anyhow. Store & database. Directories? Presume that "database" includes both LDAP and MySQL? Presuming config files, ssl certs, etc would be required aswell.

I believe openldap data is in openldap-data, ssl certs in conf, messages are in store, database is in db. Though some it may have migrated to the db.

This is the main question in this thread would be if anyone do have an exact list on what's required and used to get your zimbra working doing a zimbra install then restore of files :-)

Hopefully that exact list wouldn't change over time and new versions. Good luck.

[mhakali]

Originally Posted by LaFong

So, you want to take a full backup every time, but just of the minimum files? The average backup time will be considerably longer, and take up more room. For instance, a full backup of our server would take 70GB or so, just for the minimum files. The other files take up much less than 1GB. We do a full once a week, but the diffs during the week only take several hundred MB.

My last backup was 1.4GB tar-gzipped. Whereof I suspect 400MB or something are from redundant files already found in the .deb files which will be restored upon reinstall before restore.

Besides. Even if I did an incremental backup, it'd feel excellent shaving off the first 400-500MB of "useless" files.

Originally Posted by LaFong

Our backup software can do a bare metal restore, though I would end up copying over the /opt/zimbra backup because the dbs and un-synced message files are backed up with Zimbra stopped to insure consistency.

Cool. However not what I'm having in mind for my home mail server. :-) Would perhaps do it in a company environment. I do back up my whole vm image before I perform any version upgrades. This is probably my as-close-to bare metal backups I perform.

Originally Posted by LaFong

Only if you have to restore the entire server. If, instead, you have db corruption, you don't have to restore the whole server, just /opt/zimbra. If you did have to restore the whole server, you would reinstall Zimbra, as you've noted. Then, just copy over /opt/zimbra, done.

Yes. And this is probably doable with a lot less data backed up. I.e. binaries already supplied. etc.

Originally Posted by LaFong

The .deb files are not in /opt/zimbra, are they? I'm on CentOS, and the rpm installer is elsewhere. You don't need to back up the installer, but the binaries in /opt/zimbra are a very small portion of its overall size.

No. Not as a part of the installation. But all the raw data extracted to /opt/zimbra during your first install comes from these files.

When they are installed/extracted they are probably even larger. And this is a part of your active installation. zimbra-core is 171MB as a .deb, 411MB extracted. During a restore scenario these 411MB backed up data is probably 99% redundant and useless for a successful outcome. Which is the bare minimum I want to achieve.

Sure. While taking backup it will be compressed again. So I still state that the overhead of the total backup is about half a gigabyte.

Originally Posted by LaFong

I believe openldap data is in openldap-data, ssl certs in conf, messages are in store, database is in db. Though some it may have migrated to the db.

Ahh ok. May be the case.