Any ideas to simplify the awkward rsync expression below?

Question

Here is expression:

rsync -avz --progress -e "ssh -i /where/is/my/id_dsa" vivek@some.host:`ssh -i /where/is/my/id_dsa  vivek@some.host "find /remote/source/dir -type f -name '*.sql' | sort -nr | head -1"` .

It looks for one fresh sql file on remote host some.host in /remote/source/dir with user vivek and copies to current dir on localhost, it uses /where/is/my/id_dsa key for ssh and rsync authorization.

So this backquoted ssh -i ... frightens me a lot. But I don't see other way to find necessary file. How can I simplify it?

Thanx in advance

Answer 1

You can simplify the command by adding configuration below into your .ssh/config:

    host some.host
        identityfile /where/is/my/id_dsa
        user vivek

Now your command would be simply

rsync -avz --progress -e "ssh some.host:`ssh some.host "find /remote/source/dir -type f -name '*.sql' | sort -nr | head -1"` .

Usually I prefer --partial --progress than --progress, and it has a shorter option 'P', so your command would be

rsync -avzPe "ssh some.host:`ssh some.host "find /remote/source/dir -type f -name '*.sql' | sort -nr | head -1"` .

If you keep all sql files those has been transferred in the past in current directory, no need to choose files to be transferred selectively, because rsync will do comparison for you and won't retransfer existing files. So your command can be simplified further to

rsync -avzPe "ssh some.host:`ssh some.host "find /remote/source/dir -type f -name '*.sql'"` .

And last, you can use rsync's filter rule for your command

rsync -avzP some.host:/remote/source/dir/**/*.sql .

Please be careful to use command above, because subdirectory structure will be stripped out. Try running it with --dry-run first. Notice also that you don't need -e ssh blah, because single ':' will automatically tell rsync to use ssh instead. Let me know if this does not satisfy your need.

Cheers!

Answer 2

First, you can simplify the SSH commands by adding an alias to your ~/.ssh/config file.

Host somehost
HostName some.host
User vivek
IdentityFile /where/is/my/id_dsa

On to the main meat: you need to perform two steps, first determine which file to copy (which must be done on the remote side), then copy or update that file. It would be easier if you didn't have to worry about which files are remote and which ones are local. So mount the remote filesystem with sshfs.

mkdir somehost
sshfs somehost:/ somehost

You can now act entirely on local files. You should probably pass the --no-whole-file option to rsync, because otherwise it assumes that a local filesystem is fast and will always copy the whole file rather than apply its incremental delta algorithm.

rsync -avz --progress --no-whole-file $(find somehost/remote/source/dir -type f -name '*.sql' | sort -nr | head -1) .

Now, rather than call find, use zsh as your shell. Like most find calls, yours can be replaced by a few glob qualifiers: . for -type f, On for a numerical sort, and [1] to keep only the first match.

rsync -avz --progress --no-whole-file somehost/remote/source/dir/**/*.sql(.on[1]) .