Exclude characters for SCP-filepattern

Question

Is there a way to build an expression for scp with which I can manage to copy the file

file-name-version-1.2.1.jar

without copying the files

file-name-version-1.2.1-sources.jar
file-name-version-1.2.1-javadoc.jar

while not knowing the exact version numbers? file-name-version-*.*.*.jar won't work for obvious reasons. I'm looking for a syntax like

file-name-version-\d.\d.\d.jar

or something similar.

Answer 1

One thing to bear in mind is that the filename is passed unmodified to the login shell of the remote user.

If you do:

 scp user@remote-machine:'something' ...

On the remote machine, sshd will run the login shell of user as:

 that-shell -c 'scp -f something'

And it's that shell that will expand the patterns in there.

What that means is that the way the pattern is expanded will depend on what login shell the remote user is using. When that shell is bash, that will also depend on what the user has in his ~/.bashrc, or what there is in /etc/bash.bashrc (for instance bash-completion called from there when installed turns on ksh-style extended globbing) since for some reason bash reads those when called over ssh (even though that's not an interactive shell).

What that also means is that if something is for instance 'blah; rm -rf "$HOME"', that will have catastrophic consequences.

That's one of the misdesigns of ssh.

If you want to force a specific shell to expand that something into a list of files, you can use this trick:

LC_SCPFILES=something scp -o SendEnv=LC_SCPFILES "localhost:</dev/null
  bash -O extglob -c 'exec scp -f -- \$LC_SCPFILES';exit" /dest/dir

That only works if the login shell of the remote user is a normal Unix shell (of the Bourne, csh, rc or fish families at least), and bash is installed over there and sshd allows passing environment variables whose name starts with LC_.

The trick is that most sshd deployments allow passing environment variables whose name starts with LC_ (needed so that a Spanish user can get error messages from the remote commands in Spanish (instead of the default language on the remote system or the language of the remote user) for instance).

So here, we pass something in a LC_SCPFILES environment variable.

On the remote machine, sshd will run:

that-shell -c 'scp -f </dev/null
  bash -O extglob -c '\''exec scp -f -- $LC_SCPFILES'\'';exit'

Because its stdin is redirected from /dev/null, the first scp will exit straight away. Then we start our shell of choice (bash), with the extglob option, and with the exec scp -f -- $LC_SCPFILES command line to interpret.

Because bash will run in a child process of that-shell (which we enforce by adding a ;exit afterwards), bash will not source the ~/.bashrc, so we can expect the default behaviour of bash there (unless somehow that-shell, on startup, sets the BASHOPTS, SHELLOPTS or BASH_ENV environment variables).

Because $LC_SCPFILES is not quoted, it will be subject to word splitting and filename generation (globbing), but not the other shells expansions, and it will not do things likes rm -rf "$HOME" just because something contains it.

Now that we made sure that the pattern is expanded by bash -O extglob on the remote machine, we can use it to match a pattern that uses extended glob operators (here using a helper function):

safer_scp() (
  file=$1; shift
  export LC_SCPFILES="${file#*:}"
  exec scp -o SendEnv=LC_SCPFILES "${file%%:*}:</dev/null
    bash -O extglob -c 'exec scp -f -- \$LC_SCPFILES';exit" "$@"
)
safer_scp user@host:'file-name-version-+([0-9]).+([0-9]).+([0-9]).jar' .

The +([0-9]), with extglob on in bash matches one or more decimal digits. So the pattern above would match file-name-version-1.2.3.jar and file-name-version-12.123.1234.jar for instance.

(note that the safer_scp above assumes the first argument is the host:file-patterns, things like safer_scp -r ... or safer_scp host1:f1 host2:f2 ... won't work. If you want to use -r, the easiest is to define a safer_scp_r function where scp -f above is replaced with scp -r -f).

Note that you can disable word splitting by adding a IFS=; before the exec scp... above.

Also note that we're using bash here on the basis that it's installed on all GNU systems, but if you know zsh is installed on the remote system, you can use

zsh -o extendedglob -o globsust

instead to benefit from the full power of zsh globbing (recursive, qualifiers...)

(add -o shwordsplit if you want word splitting as well).

For example:

safer_scp() (
  file=$1; shift
  export LC_SCPFILES="${file#*:}"
  exec scp -o SendEnv=LC_SCPFILES "${file%%:*}:</dev/null
    zsh -o extendedglob -o globsubst -c 'exec scp -f -- \$LC_SCPFILES';exit" "$@"
)
safer_scp user@host:'file-name-version-<->(.<->)#.jar' .

Would copy file-name-version-1.jar and file-name-version-1.2.3.4.5.jar (<x-y> is any decimal integer number from x to y, <-> is any decimal integer number, # is like the * regexp operator (0 or more of the preceding atom))

Answer 2

In this particular case

scp 'user@host:file-name-version-*[0-9].jar' .

might be enough for you.

Answer 3

I think you want

file-name-version-[0-9].[0-9].[0-9].jar

Answer 4

For flexibility, use rsync instead of scp.

rsync --include='file-name-*.jar' \
      --exclude='*-sources.jar' --exclude='*-javadoc.jar' \
      example.com:/some/directory/ local/directory/