optimizing usage of /opt/zimbra/redolog/archive/ and rsyncing backups

[ewilen]

So if I understand correctly, when the active redolog in /opt/zimbra/redolog/redo.log gets big enough, it's moved to /opt/zimbra/redolog/archive/

Then when an incremental is run, the contents of /opt/zimbra/redolog/archive/ are moved/copied into the redolog directory inside the current incremental session directory.

Now I'm noticing two things on my system after an incremental.

• The last redolog in /opt/zimbra/backup/sessions/incr-<date-seq>/redologs/ is still in /opt/zimbra/redolog/archive/
  
• They're separate files, you can tell by inspecting their index numbers with ls -i.

So I'm wondering:

• Why isn't the redolog moved instead of copied?
  
• If there's a reason to keep it in both places, why isn't it hardlinked?

Granted, hardlinks can't traverse filesystems, so if /opt/zimbra/backup/ is on different filesystem, a simple ln won't work. But you could instead do

ln /opt/zimbra/redolog/archive/foo /opt/zimbra/archive/tmp-foo
mv /opt/zimbra/archive/tmp-foo /opt/zimbra/backup/sessions/incr-<date-seq>/redologs/foo
rm /opt/zimbra/archive/foo

Benefit: when backup and /opt/ are on the same filesystem, the incremental is less expensive, takes up a little less space, and (most important) if someone is rsyncing redologs and backups in one command, the rsync will be faster/less bandwidth-intensive.

As things are currently, you should still be able to speed up rsync using some scripting with some combination of --compare-dest, --copy-dest, and/or --link-dest, but that's messier than it ought to be.

[ewilen]

No answers...other than locating why the redologs are kept in both places : Ajcody-Backup-Restore-Issues - Zimbra :: Wiki. Still not clear why a hardlink isn't used.

So I just filed Bug 47764 - when possible, create hard links instead of copying redologs from archive to incremental backup as an enhancement.