102

I would like to transfer files between two remote hosts using on local shell, but it seems rsync doesn't support synchronisation if two remotes are specified as follow:

$ rsync -vuar host1:/var/www host2:/var/www
The source and destination cannot both be remote.

What other workarounds/commands I could use to achieve similar results?

4

16 Answers 16

104

As you have discovered you cannot use rsync with a remote source and a remote destination. Assuming the two servers can't talk directly to each other, it is possible to use ssh to tunnel via your local machine.

Instead of

rsync -vuar host1:/var/www host2:/var/www

you can use this

ssh -R localhost:50000:host2:22 host1 'rsync -e "ssh -p 50000" -vuar /var/www localhost:/var/www'

The first instance of /var/www applies to the source on host1, the localhost:/var/www corresponds to the destination on host2.

In case you're curious, the -R option sets up a reverse channel from port 50000 on host1 that maps (via your local machine) to port 22 on host2. There is no direct connection from host1 to host2.

9
  • 2
    the reverse connect does not read the ~/.ssh/config on the local side - need to use something that can be resolved as if there were no SSH config file Commented Apr 29, 2016 at 10:40
  • @Florenz the reverse connect does read the ~/.ssh/config but as the remote host is mapped as localhost on port 50000 it's probably not matching your configuration Commented Apr 29, 2016 at 11:22
  • @roaima that was more a "mental note to self". It reads it's config, on the remote server, so alias configuration is to be interpreted in the scope of the executing host, it's the remote's shell that is running here after all Commented May 3, 2016 at 7:59
  • @FlorenzKley oh I see what you mean. I'd already mentally swapped "remote" and "local" so that the words were used in the context of the reverse connect. Yes, you're right: in the context of the original connection, the reverse tunnel can only use the "remote" host's ~/.ssh/config file. Commented May 3, 2016 at 9:15
  • 2
    'Assuming the two servers can't talk directly to each other'. This solution does work around a firewall or NAT issue that prevents a direct SSH connection. However, it doesn't address the case where for security reasons the source user (on host1) has no key or credentials or insufficient write permissions on the destination. For that see Kevin Cox's solution, or resort to an indirect connection using a script or scp -3. Commented Aug 21, 2017 at 8:15
41

You didn't say why you didn't want to log into one host and then copy to the other so I will share one of my reasons and solutions.

I couldn't log into one machine then rsync to the other because neither host had a SSH key that could log into the other. I solved this by using SSH agent forwarding to allow the first host to use my SSH key while I was logged in.

WARNING: SSH forwarding allows the host to use your SSH key for the duration of your login. While they can't copy your key they can log into other machines with it. Make sure you understand the risks and don't use agent forwarding for machines you don't trust.

The following command will use SSH agent forwarding to open a direct connection from host1 to host2. This has the advantage that the machine running the command isn't bottlenecking the transfer.

ssh -A host1 rsync -vuar /var/www host2:/var/www
6
  • 5
    +1 for explaining a valid use case (where the remote user on host1 has no permissions on the destination server); for the important security caveat (use port forwarding -D rather than -A to get around network rather than key restrictions); for explaining the advantage; for the command being short; and it actually working. Note that you may need to specify username@host1 if it's different from local username. Also that rsync performs host key verification when connecting to host2, so host1's key should already be in ~/.ssh/known_hosts on host2 or the command will fail. Commented Aug 21, 2017 at 8:21
  • Phenomenal answer, this helped me orchestrate some things in TeamCity that I wasn't able to do before (n.b. for other TeamCity users, you must add the "Build Feature" called "SSH agent" to your build config before using ssh -A, see confluence.jetbrains.com/display/TCD10/SSH+Agent). Commented Mar 7, 2018 at 10:15
  • 1
    You forgot to mention that this solution require host1 to have direct network access to host2, which a lot of the times is not the case.
    – logicor
    Commented Aug 19, 2021 at 1:52
  • 1
    This worked best for me. I did need to add my ssh key to the SSH Agent with ssh-add first.
    – Matthias
    Commented Aug 26, 2021 at 16:20
  • @Kevin Cox >"why you didn't want to log into one host and then copy to the other" . This did the trick for me, thanks a lot! :)
    – Tms91
    Commented Apr 15, 2022 at 9:12
22

I like roaima's answer, but the paths are the same in both examples, obscuring which is which. We've established that the following doesn't work:

rsync -vuar host1:/host1/path host2:/host2/path

But this does (I omitted the explicit bind_address of localhost from the -R option since that's the default):

ssh -R 50000:host2:22 host1 'rsync -e "ssh -p 50000" -vuar /host1/path localhost:/host2/path'

Note that you'll have to have ssh keys set up correctly between the two remote hosts, with the private key on host1 and the public key on host2.

To debug the connection, break this into two parts and add verbose status:

localhost$ ssh -v -R 50000:host2:22 host1

If this works, you'll have a shell on host1. Now try the rsync command from host1. I recommend doing this in a different window so the verbose ssh info doesn't get mixed together with the rsync status info:

host1$ rsync -e "ssh -p 50000" -vuar /host1/path localhost:/host2/path
2
  • In my example the paths are source, destination. The rsync is initiated on host1 with the target on host2. (You could have asked for clarification in a comment.) Commented Jan 14, 2016 at 0:12
  • 1
    I would have commented, but you can't comment on somebody else's post without a reputation of 50+.
    – jaybrau
    Commented Jan 14, 2016 at 15:20
16

Reformatting answer by roaima in bash script syntax (and adding line continuation characters '\' for clarity) I randomly picked port 22000...

SOURCE_USER=user1
SOURCE_HOST=hostname1
SOURCE_PATH=path1

TARGET_USER=user2
TARGET_HOST=host2
TARGET_PATH=path2

ssh -l $TARGET_USER -A -R localhost:22000:$TARGET_HOST:22 \
$SOURCE_USER@$SOURCE_HOST "rsync -e 'ssh -p 22000' -vuar $SOURCE_PATH \
$TARGET_USER@localhost:$TARGET_PATH"
5
  • 2
    it seems to me that all you've done is replace his arbitrary hostnames with variables?
    – Jeff Schaller
    Commented Oct 20, 2016 at 23:18
  • 6
    Yes, I did. It adds clarity for me on which is the source machine, which is the target, and where the source and target paths go. It took me awhile to work all that out and it wasn't obvious from simple placeholder hostnames.
    – David I.
    Commented Oct 25, 2016 at 15:23
  • Next time please feel free to improve someone else's answer directly by editing it. Commented Apr 11, 2017 at 7:30
  • 1
    This answer was the solution for me as it combines ssh-agent forwarding (-A) with the reverse tunnel (-R). Commented May 2, 2019 at 9:46
  • it also clarifies where to put user1 and user2!
    – toto_tico
    Commented Mar 20, 2020 at 16:52
7

An easy to use script

Over the years I've done this many times with more or less the same tricks as in every other answer here. However because it's very easy to get some detail wrong and spend a lot of time figuring out the issue I've come up with the script below that:

  1. Makes it easy to specify all the details (source, destination, options)
  2. Incrementally tests every single step and gives feedback if anything goes wrong, so that you know what to fix.
  3. Works around cases where ssh -A fails to propagate authentication data (don't know why this happens sometimes since the workaround was easier than finding the root cause)
  4. Finally does the job.

How to use the script

  1. Make sure you can ssh to both hosts from localhost without typing a password.
  2. Set the variables in the first few lines of the script
  3. Execute it.

How it works

As I said it uses the same tricks as in every other answer here:

  • ssh's -R option to ssh from localhost to host1 while at the same time setting up a port forwarding that then allows host1 to connect via localhost to host2 (-R localhost:$FREE_PORT:$TARGET_ADDR_PORT)
  • ssh's -A option to allow easy authenticaion of the second ssh chanel

My this IS complicated! Is there any easier way?

When copying all or most bytes from source to destination it's FAR easier to use tar:

ssh $SOURCE_HOST "tar czf - $SOURCE_PATH" \
    | ssh $TARGET_HOST "tar xzf - -C $TARGET_PATH/"

The script

#!/bin/bash
#-------------------SET EVERYTHING BELOW-------------------
# whatever you type after ssh to connect to SOURCE/TARGE host 
# (e.g. 1.2.3.4:22, user@host:22000, ssh_config_alias, etc)
# So if you use "ssh foo" to connect to SOURCE then 
# you must set SOURCE_HOST=foo
SOURCE_HOST=host1 
TARGET_HOST=host2 
# The IP address or hostname and ssh port of TARGET AS SEEN FROM LOCALHOST
# So if ssh -p 5678 [email protected] will connect you to TARGET then
# you must set TARGET_ADDR_PORT=1.2.3.4:5678 and
# you must set TARGET_USER=someuser
TARGET_ADDR_PORT=1.2.3.4:5678
TARGET_USER=someuser

SOURCE_PATH=/mnt/foo  # Path to rsync FROM
TARGET_PATH=/mnt/bar  # Path to rsync TO

RSYNC_OPTS="-av --bwlimit=14M --progress" # rsync options
FREE_PORT=54321 # just a free TCP port on localhost
#---------------------------------------------------------

echo -n "Test: ssh to $TARGET_HOST: "
ssh $TARGET_HOST echo PASSED| grep PASSED || exit 2

echo -n "Test: ssh to $SOURCE_HOST: "
ssh $SOURCE_HOST echo PASSED| grep PASSED || exit 3

echo -n "Verifying path in $SOURCE_HOST "
ssh $SOURCE_HOST stat $SOURCE_PATH | grep "File:" || exit 5

echo -n "Verifying path in $TARGET_HOST "
ssh $TARGET_HOST stat $TARGET_PATH | grep "File:" || exit 5

echo "configuring ssh from $SOURCE_HOST to $TARGET_HOST via locahost"
ssh $SOURCE_HOST "echo \"Host tmpsshrs; ControlMaster auto; ControlPath /tmp/%u_%r@%h:%p; hostname localhost; port $FREE_PORT; user $TARGET_USER\" | tr ';' '\n'  > /tmp/tmpsshrs"

# The ssh options that will setup the tunnel
TUNNEL="-R localhost:$FREE_PORT:$TARGET_ADDR_PORT"

echo 
echo -n "Test: ssh to $SOURCE_HOST then to $TARGET_HOST: "
if ! ssh -A $TUNNEL $SOURCE_HOST "ssh -A -F /tmp/tmpsshrs tmpsshrs echo PASSED" | grep PASSED ; then
        echo
        echo "Direct authentication failed, will use plan #B:"
        echo "Please open another terminal, execute the following command"
        echo "and leave the session running until rsync finishes"
        echo "(if you're asked for password use the one for $TARGET_USER@$TARGET_HOST)"
        echo "   ssh -t -A $TUNNEL $SOURCE_HOST ssh -F /tmp/tmpsshrs tmpsshrs"
        read -p "Press [Enter] when done..."
fi

echo "Starting rsync"
ssh -A $TUNNEL $SOURCE_HOST "rsync -e 'ssh -F /tmp/tmpsshrs' $RSYNC_OPTS $SOURCE_PATH tmpsshrs:$TARGET_PATH"

echo
echo "Cleaning up"
ssh $SOURCE_HOST "rm /tmp/tmpsshrs"
4
  • tar is great when you've got a single (non-incremental) transfer and your transfer completes in a single pass. On the other hand, rsync with forwarding handles restarts and incremental transfers. Commented Apr 4, 2019 at 12:34
  • 1
    Of course @roaima -- I don't consider tar equivalent. I left this passing reference for the day when I will be reading this to solve a problem where rsync will not 100% necessary.
    – ndemou
    Commented Apr 4, 2019 at 13:29
  • Massive kudos for this, I was looking at scripting this manually, so this is super handy to have as an existing bash script Commented Aug 18, 2021 at 9:15
  • This deserves to be a full fledged package for common distros. Commented Aug 19, 2023 at 8:49
3

The ideal way would be to run the rsync on one of those servers. But if you do not want to run a script on the remote server. You could run a script on your local system and do an ssh and execute the rsync there.

ssh user@$host1 <<ENDSSH >> /tmp/rsync.out 2>&1
rsync -vuar /var/www host2:/var/www
ENDSSH

Also, as you maybe aware rysnc does one way synchronization . If you'd like two way synchronization , you can look at osync (https://github.com/deajan/osync). I use it and found it to be helpful.

1

You could use a local temporary directory to copy the data from the remote source server and then copy this temp to remote destination server.

An example is this:

SRC=user@source-server:/some/path/in/source/* && \
DST=user@destin-server:/some/path/in/destin/* && \
TMP=/tmp/rsync && \
rm -fr $TMP && \
mkdir -p $TMP && \
rsync -av $SRC $TMP && \
rsync -av $TMP/* $DST/ && \
rm -fr $TMP
1

Not sure if this is of any use to anyone... This is the quick and nasty solution I came up with to sync files between two remote systems that the host is authorized to access, but where the remotes cannot access each other:

(){ local RSYNC_TMP=$(mktemp); rsync -aP src-host:~/filename $RSYNC_TMP; rsync -aP $RSYNC_TMP dest-host:~/filename; rm -rf $RSYNC_TMP }

This can be copy/pasted into zsh, so good for use if your host is macOS. This is not going to keep the file in question safe, however. Swapping out the file name for a directory name, and changing local RSYNC_TMP=$(mktemp) to local RSYNC_TMP=$(mktemp -d) should allow for directory transfer.

For a more secure temporary transfer mechanism, provided that your host is Linux, I'd be inclined to use [bwrap(1)][1] or somethibng like that to create a tmpfs mount that only lives for the duration of the process.

0

You could run an rsyncd (server) on one of the computers.

This is the approach I'm taking, since I do not want to use ssh to allow the 'source' (in rsync wording) access to the 'destination' as root without a password (as is required to use SSH tunneling with rsync in a script)

In my case, I simply set up an rsyncd server on the destination computer with a single user allowed from the source pc and used rsync from the source side.

Works great.

0

If you are not able to run rsync on at least one of the servers (host1 or host2) and you don't care about bandwidth and performance, I always do it like this (on a third server, not local) as it is super easy to remember:

$ mkdir host1 host2
$ sshfs user1@host1:/path1 ./host1
$ sshfs user2@host2:/path2 ./host2
$ rsync -a ./host1/ ./host2/

If you do it this way (with sshfs), you won't need any extra disk space on the server where you run these commands. If you don't care about disk space either, you can simply do it like this (without sshfs):

$ mkdir host1
$ rsync -a user1@host1:/path1/ ./host1/
$ rsync -a ./host1/ user2@host2:/path2/

Of course you can change the rsync parameters as you like. I mostly just use -a.

1
  • sshfs is the path I'm coming from. Unfortunately, sshfs can't handle hardlinks at the source, so be wary which data you are copying over it. Commented Jan 19, 2023 at 8:01
0

Having not the reputation points required, I can't add a comment on Chris Davies's answer. However, I don't think everybody will be able to have his solution working without some help.

ssh -R localhost:50000:host2:22 host1 'rsync -e "ssh -p 50000" -vuar /var/www localhost:/var/www'

means that you ssh to host1 while building a tunnel to host2 trough the intermediate server. And on host1, you execute the rsync command using ssh to reach the target machine. If you are unlucky, it won't work and if you slightly modify the command, you'll know why :

ssh -R localhost:50000:host2:22 host1 'rsync -e "ssh -vv -p 50000" -vuar /var/www localhost:/var/www'

You'll get messages such as :

debug1: load_hostkeys: fopen /home/ubuntu/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory

This means that .ssh/known_hosts was found on host1 but since the destination is not known from ssh, it attempts to find it elsewhere. And since you are not in an interactive shell, it can't ask the question :

The authenticity of host 'host2 (host2)' can't be established.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

To solve this, the simplest way is to build a temporary tunnel :

ssh -R localhost:50000:host2:22 host1 'sleep 60'

and then get to host1 and from there, run the ssh command :

ssh -p 50000 localhost

(which means "ssh host2") in order to validate the host. This being done, the accepted solution should be working.

Nota : this question should be merged with rsync files between two remotes, with only access from the "intermediate"

0

Improving on Chris Davies:

  • add usernames
  • add no strict host key checking, in case host1 doesn't know 'localhost' yet.
  • create tunnel to host2 separately for special cases
  • add batchMode, to not let script hang on password if key authentication fails

instead of

rsync -vuar user1@host1:/var/www user2@host2:/var/www

you can use this

ssh -o BatchMode=yes -R localhost:50000:host2:22 user@host1 'rsync -e "ssh -p 50000 -o StrictHostKeyChecking=no -o BatchMode=yes" -vuar /var/www user2@localhost:/var/www'

And if access to host2 needs special treatment, e.g. hop via other hosts, then you can separate the tunnel creation:

ssh -o BatchMode=yes -o ExitOnForwardFailure=yes -f -N -L 60000:host2:22 someuser@somemmachinethatcanreachhost2
ssh -o BatchMode=yes -R localhost:50000:localhost:60000 user@host1 'rsync -e "ssh -p 50000 -o StrictHostKeyChecking=no -o BatchMode=yes" -vuar /var/www user2@localhost:/var/www'
pkill '^ssh$' #careful with this one. Or:
SSH_PID=$(ps aux | grep "ssh -o BatchMode=yes -o ExitOnForwardFailure=yes -f -N -L 60000:host2:22 someuser@somemmachinethatcanreachhost2" | grep -v grep | awk '{print $2}')
kill $SSH_PID
-1

It's possible to use tar via ssh to transfer the files:

ssh -n user1@host1 'tar jcf - -C /var/www .' | ssh user2@host2 'tar jxvf - -C /var/www'

Change j parameter (for tar) to z in two places if you wish to compress the archive with gzip, instead of bzip2. Usually bzip2 has the higher compression than gzip, but it's slower, so change it depending on your needs (see: bzip2 vs gzip).

Related: How to copy between two remote hosts using tar piped into SSH from remote server when behind a firewall?


Alternatively (to safe the bandwidth, because of the transparent compression) it's possible to use sshfs to mount remote file-system as local and use rsync as usual, e.g.

$ sshfs user1@host1:/var/www /mnt
$ rsync -vuar /mnt user2@host2:/var/www
1
  • 1
    Mounting the filesystem locally wont save any bandwidth between the source and local machine. It will however save bandwidth between the local and destination (as it isn't mounted locally). Using rsync this way only makes sense if you are trying to save bandwidth on the destination.
    – Kevin Cox
    Commented Dec 3, 2015 at 14:31
-1

Try using this. It works for me.

ssh src_user@src_host 'rsync -av /src/dir/location/ dest_user@dest_host:/dest/dir/loc/'
1
  • Only works if you can go direct from src machine to destination. Whcih is NOT what the OP wanted
    – anthony
    Commented Feb 14, 2023 at 5:37
-1

I think this should work :

ssh user@host1; "(cd /somedir; tar czf -)" | ssh ssh user@host2; "(cd /somedir; tar xzf -)"

2
  • Does it actually work? Have you tested it? There's already a working, simpler tar option given by the OP: unix.stackexchange.com/a/183509/70524
    – muru
    Commented Aug 15, 2023 at 1:47
  • Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
    – Community Bot
    Commented Aug 15, 2023 at 15:06
-2

Just as an additional info:

If you use a jump host to connect the other two machines but they cannot reach each other directly, you can use sshfs as medium between these two machines like so (on the jump host):

$ mkdir ~/sourcepath ~/destpath
$ sshfs sourcehost:/target/dir ~/sourcepath
$ sshfs desthost:/target/dir ~/destpath
$ rsync -vua ~/sourcepath ~/desthpath

SSHFS provides the two paths on the jump host and rsync manages the synchronisation of the files like always (just with the difference that it's virtually done locally).

1
  • 2
    Note that the performance will be terrible. This is because to detect changes rsync will read the file from the source server, transferring the whole thing over the network. That being said if you can't do a direct transfer you will have to eat this if you are using rsync. Also don't mount the destination, it is unnecessary and will cause rsync to change some defaults because it thinks it is talking to a local disk.
    – Kevin Cox
    Commented Aug 22, 2016 at 18:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .