I'm not sure what options you have available to you, but I've always been a big fan of simple is better.
We accomplish off-site backups using
rsync. rsync is capable of deltas even on binary files. It will save a ton of time. You can use this over SSH so that it's encrypted (recommended) and using host keys you won't need to supply a password either. This can be initiated by the backup client or the backup server.
Once the files are at the remote location, you can do whatever you need to them.
I'll leave it up to your research if you feel you should shut down certain services or not before a backup. You definitely do not want to restore to a live environment. Meaning if you're backing up MySQL to a remote server, MySQL shouldn't be running on that remote server, or it should be dumping to a folder that's outside of MySQLs view. It should go without saying that backing up a live MySQL box probably isn't the best idea either.
For that, we cluster our production MySQL server with a read-only MySQL server. We then shut down that read-only server and make a backup of it. Works pretty well and it's all automated through simple shell scripts.
My recommendation is to create folders for each server you have on your off-site server. Use rsync to keep those folders and servers synchronized. Then run your backups (tape or whatever it may be) from that off-site servers view of the server in those folders.
Another great tool, which uses the rsync libraries, is
rdiff-backup. I highly recommend this tool. It does the versioning for you automatically, restoring something from 10 days ago into a temp location is as simple as:
rdiff-backup -r 10D host.net::/remote-dir/file /tmp/file
If you just need the most recent backup, you can simply copy it from the remote host.
-Bill