Compressing redologs during incremental backups?

[nate.olmstead]

I'm using the -z (or --zip) option to compress message blobs during backup. This is very useful for a number of reasons, especially regarding performance of sending tons of small files over NFS as discussed in the 49,700 success stories thread. However, it appears this pertains to message blobs only, and not redolog files, which appear to archive at about the 100MB mark by default.

Is there anything in the workings to get redologs compressed in some manner? Compressing them manually would likely break PITR. It would be great to have that as an option. Unfortunately, just keeping the latest set of redologs isn't an option as some customers want to retain a somewhat granular PITR for an extended period, such as restoring to a particular day and not a particular week. The CPU cycles to compress the files would be of negligible impact in this case.

The other option I'm considering is just writing a nanny script on the backup server to compress incremental backups older than a practical threshold (7-14 days) and compress them manually. Of course, they would have to be manually expanded again prior to doing a PITR from that period of time. The disk space savings might be worth the very occasional inconvenience of manually expanding the files again for a restore, however. Initial tests show about a 35-40% compression of the files using gzip with no arguments. bzip2 -9 might do even better.

Just looking for ideas, tips from existing implementations, etc.

[rainer_d]

You could store them on an NFS-server and use OpenSolaris+ZFS+on-the-fly-compression for the NFS-server.
;-)
If they compress at all.

[nate.olmstead]

They do compress, at least 35% or so from the initial testing I did, using gzip. I did consider ZFS, but, this feature isn't important enough to want to rebuild a backup server with OpenSlowlaris and ZFS and deal with maintaining yet one more OS in house. Reiser4 may be viable but... well, I'm just not going to go there.

The jar that does the backups works well restoring from zip files and I was just hoping there was a plan in the works to allow it to do the same for the redologs. Guess I'll look at an interim solution, like the nanny script mentioned earlier.