Backups - RSYNC & copying files issue

[sternfan]

Hi All,

Scenario: I backup my zimbra server using RSYNC to a NFS share (another server named BACKUP next to it, connected using a switch over gigabit). I do the RSYNC and looks good - no errors at the end that I can tell.

I have another server (I call ARCHIVE) that is in another building (off site) but still connected by gigabit network. My standard procedure is to do the backup to the backup server, then copy this backup to the ARCHIVE server (off site backups done the easy way).

OK - so I go to copy the backup folder to ARCHIVE and a whole bunch of files will not copy. I get an ERROR "UNSUPORTED OPERATION..." then the filename. There are over a hundred of these. The first couple are:

rammd5.so
libplain.so
login.so.z
ssapiv2.so

and on and on...

Once done - I look at the properties of both folders:

Backup is 26.1g with 313,273 files.
Archive is 26.1g with 312,990 files.

So I am basically missing about 283 files after the copy operation (they won't copy).

First Question - what can cause this? The fact that ARCHIVE is an SMB share? Something to do with permissions? RSYNC? I've copied larger files to the SMB share before with no problems.

Second Question - I tried to move my Zimbra install to new hardware about a week ago using roughly the same procedure (RSYNC to BACKUP, then on the new server RSYNC from BACKUP etc). I couldn't make this work and am wondering if this could be related to why.

Thanks for any and all help,
Rob

[raj]

Hi..did you follow the WIKI
Open Source Edition Backup Procedure - Zimbra :: Wiki

you did not mention if you RSYNC'ed while zimbra was runnnig or you stopped all services.
I am assuming you running OSS Version and Yout rying to backup files while zimbra is running..can you please give more info on this.

Basic steps to do COLD backup on same machine (/opt/zimbra_backup ) would be

As zimbra user:
~ su - zimbra
~ zmcontrol stop
~ logout

As root user:
# rsync -avHK /opt/zimbra/ /opt/zimbra_backup/zimbra

As zimbra user:
~ su - zimbra
~ zmcontrol start

Raj

[sternfan]

sorry about that - I stopped the services as normal. Verified with grep.

I did an rysnc -avr; how is that different than a rsync -avHK? I googled for it and can't find any info.

Rob

[raj]

Hi..i just used from the WIKI links and it works..what exactly it dose is I dont know..may be WIKI author can shed some light.
All I know it backups up files and permissions and any syslinks = backup zimbra correctly for cold backup and restore

i use same command to restore from backuped up folder..

Raj

[sternfan]

hi raj,

Thanks for the reply. Just to be clear - I believe rsync is working OK. I can successfully rsync my zimbra box to my backup server. It's when I then try to copy this backup to another server that for some reason the files won't copy. Weird.

Thanks,
Rob

[raj]

ok lets check this step by step...iam not rsync expert , but lets think logically.

You have ZIMBRA, BACKUP and ARCHIVE servers 3 in total
1) ZIMBRA --> BACKUP - works
2) BACKUP --> ACRCHIVE - dont work

i dont know how RSYNC handels user permissions and syslinks.
What happens if you don’t have SAME original files to syslink and SAME USER on ACRCHIVE..dose RSYNC loose all that info or it saves that into in some metadata.
What iam tying to get to is if ARCHIVE server is NOT of same OS type and don’t have "zimbra" users and group..how will it preserve file permissions by user and other properties..
May be your ZIMBARA and BACKUP server are same but ARCHIVE server is not...

just logically thinking. May be some RSYNC expert can jump in and explain..

Raj

[andreychek]

I did an rysnc -avr; how is that different than a rsync -avHK? I googled for it and can't find any info.

I'm under the impression that smbfs doesn't handle links very well, there's an excellent chance that's where you're backup is going awry. NFS would handle them better.

As for the difference between the above commands, here's some info from the rsync manpage:

-a, --archive archive mode; same as -rlptgoD (no -H)
-K, --keep-dirlinks treat symlinked dir on receiver as dir
-H, --hard-links preserve hard links

So the "r" you were using is actually redundant with the "a" option. Meaning the only difference between the two commands is the "K" and "H", which describe how to handle links.

But, if the problem is smbfs's link support, that won't help in this case :-)

What you might consider looking into is if you could try using CIFS rather than SMBFS. There is a kernel module for CIFS, you'd just want to use it over the SMBFS module.

I'm not sure if that entails any additional software installs, but poking around on google for your distro and cifs should probably help.

Good luck!
-Eric

[sternfan]

Hi all - thanks for the help.

Before I got back to the board, I setup a new NFS share and am using it now (if it works, SMB was the culprit). It's a 26g file, so it's going to take a while.

Eric - thanks for the info. So in plain language - what's the difference between rsync -avr & rsync -avHK? Just taking notes...

Thanks again,
Rob

[sternfan]

Learn something new every day... If I copy using SMB, I get errors. If I copy using NFS, no problems.

Thanks,
Rob