rsync

wp:rsync (Remote Synchronization) is a utility for efficiently transferring and synchronizing files across computer systems, by checking the timestamp and size of files; and optionally a checksum comparison. It is commonly found on Unix-like systems and functions as both a file synchronization and file transfer program. The rsync algorithm is a type of delta encoding, so that only file differences are transmitted across the network. Zlib may be used for additional compression, and SSH or stunnel can be used for data transport security.

rsync was first created by Andrew Tridgell and Paul Mackerras in 1996 .

• rsync through a firewall
• Andrew Tridgell's 1999 PhD thesis Efficient Algorithms for Sorting and Synchronization includes 3 chapters on rsync.

Forward SSH Agent, and switch to different user

Suppose you have 3 hosts:

1. your workstation
2. machine A
3. machine B

Machine A is configured so that root login is not allowed - even key-based logins (too many stupid breakin attempts). So, you've created a user 'Dan'. Dan has full sudo privileges.

Machine B is configured to allow root login, and all of Dan's public keys are listed in the authorized_keys file for /root/.ssh/authorized_keys

You want to rsync files from Machine A to Machine B, but some of those files are root-owned backups etc. A regular rsync will fail to read some files. You don't want to chmod or chown anything. Of course you start off by setting up your ssh-agent and adding your ssh keys eval $(ssh-agent) && ssh-add before connecting and forwarding your agent: ssh -A dan@machineA

Now, here's the special part: Without even needing to reconfigure sshd on Machine A, you can simply --preserve-env=SSH_AUTH_SOCK in your sudo rsync command so that using sudo doesn't break the agent forwarding for the rsync.

(On machineA)

sudo --preserve-env=SSH_AUTH_SOCK rsync -vrz --checksum /var/discourse root@machineB:/var will successfully read all files on MachineA using elevated privileges of sudo, and transfer them to MachineB

Note: to make a permanent configuration in sshd_config, you'd use something like this:

Defaults>root    env_keep+=SSH_AUTH_SOCK

Jump through bastion host using ssh

If you're working on a remote terminal and your repository get's complicated, but you don't have desktop tools like meld or gitk on the remote host to examine things, you can copy the remote repo to your desktop with rsync, even if you have to jump through a bastion host with something like:

rsync -e "ssh -t bastion ssh -A" -ravz centos@10.0.50.68:/opt/meza/ ./meza-es1/

If that complains about host-key verification, then simply do an SSH first to the bastion host, accept the host key identity, and logout. Now the rsync will work because the host-key is already accepted as valid.