2

Thanks to the sharp eye of @roaima it has come to my attention that my remote server backup script below, does NOT actually use the SSH encryption as I would like it to.

I based my rsync command on the example found here:
https://www.man7.org/linux/man-pages/man1/rsync.1.html

rsync -av -e "ssh -l ssh-user" rsync-user@host::module /dest

However, what is actually happening when my current backup script executes is:

1.) it connects to ssh as the designated user, then
2.) my login notification script confirms the ssh user's login with an email (repeats on disconnect/re-connects),
3.) PROBLEM: The rsync daemon connects and does its business -outside- the ssh shell, meaning I don't actually get the ssh shell encryption that I want.

I verified this by executing the backup script and then on the remote server executing the who command, which confirms that the ssh-backup-user is -not- connected to the server while the rsync daemon is executing.

My current backup script (has to executed as non-root user)

#!/bin/bash
while [ 1 ]
do
     rsync -avxP --delete --append --checksum --timeout=180 --bwlimit=150 --rsync-path="sudo rsync" --log-file=/var/log/rsync.log --password-file=/etc/rsyncd.passwd -e "ssh -l backup-user" 111.22.333.444::data /media/user/WebMade/Server-Backups/Prod/today/
    if [ "$?" = "0" ] ; then
        echo "rsync completed normally"
        exit
    else
        echo "Rsync failure. Backing off and retrying..."
        sleep 10
    fi
done
#EOF

Might someone be able to clarify whats happening, either I am misunderstanding the man page (most likely :-/ ) or the provided example is wrong.

Thanks

3
  • Not an answer, but if you're primary objective is doing backups (instead of having the same files accessible at a second location), you should rather use something like borgbackup.org. Jul 24, 2022 at 14:49
  • 1
    "I verified this by [...] executing the who command, which confirms that the ssh-backup-user is -not- connected" - who does not confirm that, because it doesn't list non-interactive SSH sessions such as those used by rsync. Even if rsync were transfering data over SSH, you would not see it in that way. who is not the appropriate tool to check for this. Then how? Good question; at times I've verified this with top. rsync transfering with 80MB/s? ssh consuming 60% CPU? Seems to check out.
    – marcelm
    Jul 24, 2022 at 19:18
  • @marcelm I get the scenario, someone else incorrectly told me that my correct rsync command script was wrong, then rather than admit their own error everything got exacerbated into this rabbit-hole of opinion and confusion post. Compounded by my own ignorance of the who command. In any case, thx for your feedback all has been sorted out... 'tis life moving on. Jul 26, 2022 at 5:46

2 Answers 2

5

Everything is just fine, there's no problem here.

The other answers here are partially wrong and partially missing the point. What you have done is the correct documented way to spawn and communicate with the rsync daemon through an SSH connection. For those unfamiliar with this mode, I recommend checking out the section of the rsync manual called "USING RSYNC-DAEMON FEATURES VIA A REMOTE-SHELL CONNECTION".

Setting --rsh or -e to ssh in combination with the double-colon :: syntax enables this mode. Rsync will then connect to the remote machine using ssh, spawn a daemon there (under the SSH session) and communicate with it through the SSH tunnel. This is a very useful mode because it lets you use daemon-only features together with SSH authentication and encryption.

who is not telling you what you think it does. who (just as w and other alternatives) only lists interactive sessions, the ssh session used to launch the rsync daemon is not interactive (doesn't even use a shell at all), so it won't appear here. If you don't just trust rsync to do what it says in the manual, you can use ps -efH to see the ssh session and the rsync daemon running on the remote machine, ss -t to see there's no TCP connection between the machines apart from the SSH on port 22, or strace on the rsync process to see the rsync process communicating with the ssh process.

1
  • Thanks for finally providing an efficient reply that directly answers the question(s) being asked in the original post. You just saved me hours of time. Highly appreciated. Jul 24, 2022 at 23:06
1

The presence of a double colon (::) in a source of target path indicates use of the rsyncd service. A single colon (:) indicates a login (typically) over ssh.

Once you do that, the remote path must then either start with / or it's relative to the home directory the account would log in to.

For example

[email protected]:docroot

Remove --checksum and let rsync decide when to use one. (The only time it automatically skips a checksum is when the file size and modification time match on both sides.)

Stop using any variation of --append as otherwise you will end up with data corruption in your backups sooner or later. You will almost certainly have wanted --partial here, and that's already included in -P.

For a ssh login backup-user on the remote system 203.0.113.1 that has (say) two directories logs and website, you would then use a command line like this to backup:

 rsync -aivP --delete [email protected]:website/ /my/local/copy

I now understand from a comment that you need to copy files from an rsyncd service but you want to transfer them across an ssh connection that provides a secure (encrypted) transport. Conveniently the man page for rsync actually has this an an example: you set the environment variable RSYNC_CONNECT_PROG to define the ssh tunnel and then connect to the rsyncd service within the command itself:

RSYNC_CONNECT_PROG='ssh [email protected] nc %H 873' rsync -aivP --delete localhost::data/ /my/local/copy

Be careful, though. Neither solution gives you no long-term protection against a corruption such as a web page hijacked for a trojan. You replace your backup with a full copy of live each time, so a mistake on live will propagate to your backup as soon as you run your backup script. Consider GFS or some other multilevel backup schedule (three different backups, one each for Sunday, Tuesday, Thursday would be a start). A tool such as rsnapshot can implement this efficiently as a layer above rsync.

13
  • this is a great answer & accurate solution. However, I absolutely want to use the rsync daemon over ssh rather than the normal rsync because of the resource benefits. I also have to do this as ssh non-root user, so perhaps I need to play with /etc/rsyncd.conf and backup users .sshauthorized_keys settings further and make a new post if necessary. Overall what I am trying to achieve is to connect with a non-root user, then have either the non-root user or root connect to rsyncd. Perhaps this is worthy of separate post after some testing. Jul 24, 2022 at 8:29
  • I misspoke... the -delete option did in fact delete a partial file that existed on target not on source, but after a very very very long pause... (large file) it seems that rsync is respecting the --partial flag.... so your advice about removing --checksum and --append is dead on... thanks again Jul 24, 2022 at 8:43
  • "the -delete option did in fact delete a partial file that existed on target not on source" - The --delete option affects the target; the complementary --remove-source-files option turns rsync's effective action from "copy" to "move". Jul 24, 2022 at 9:50
  • "I absolutely want to use the rsync daemon over ssh rather than the normal rsync because of the resource benefits" that doesn't make sense (to me). The daemon is a system service. Rsync over ssh is a service that's invoked on demand. "I am trying to [...] connect [with ssh] with a non-root user, then have either the non-root user or root connect to rsyncd" - that's not what I understood from your question. I'll either suggest a duplicate or give you a solution for that Jul 24, 2022 at 9:54
  • 1
    I think the answer in this comment is correct. Don't use :: if you don't want to use port 873. Test by disabling the rsyncd service on port 873.
    – user10489
    Jul 24, 2022 at 13:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .