26

ssh has an annoying feature in that when you run:

ssh user@host cmd and "here's" "one arg"

Instead of running that cmd with its arguments on host, it concatenates that cmd and arguments with spaces and runs a shell on host to interpret the resulting string (I guess that's why its called ssh and not sexec).

Worse, you don't know what shell is going to be used to interpret that string as that's the login shell of user which is not even guaranteed to be Bourne like as there are still people using tcsh as their login shell and fish is on the rise.

Is there a way around that?

Suppose I have a command as a list of arguments stored in a bash array, each of which may contain any sequence of non-null bytes, is there any way to have it executed on host as user in a consistent way regardless of the login shell of that user on host (which we'll assume is one of the major Unix shell families: Bourne, csh, rc/es, fish)?

Another reasonable assumption that I should be able to make is that there be a sh command on host available in $PATH that is Bourne-compatible.

Example:

cmd=(
  'printf'
  '<%s>\n'
  'arg with $and spaces'
  '' # empty
  $'even\n* * *\nnewlines'
  "and 'single quotes'"
  '!!'
)

I can run it locally with ksh/zsh/bash/yash as:

$ "${cmd[@]}"
<arg with $and spaces>
<>
<even
* * *
newlines>
<and 'single quotes'>
<!!>

or

env "${cmd[@]}"

or

xterm -hold -e "${cmd[@]}"
...

How would I run it on host as user over ssh?

ssh user@host "${cmd[@]}"

obviously won't work.

ssh user@host "$(printf ' %q' exec "${cmd[@]}")"

would only work if the login shell of the remote user was the same as the local shell (or understands quoting in the same way as printf %q in the local shell produces it) and runs in the same locale.

  • 3
    If the cmd argument was /bin/sh -c we would end up with a posix shell in 99% of all cases, wouldn't we? Of course escaping special characters is a bit more painful this way, but would it solve the initial problem? – Bananguin May 25 '15 at 20:16
  • @Bananguin, no if you run ssh host sh -c 'some cmd', same as ssh host 'sh -c some cmd', that has the login shell of the remote user interpret that sh -c some cmd command line. We need to write the command in the correct syntax for that shell (and we don't know which it is) so that sh be called over there with -c and some cmd arguments. – Stéphane Chazelas May 25 '15 at 20:21
  • 1
    @Otheus, yes, the sh -c 'some cmd' and some cmd command lines happen to be interpreted the same in all those shells. Now what if I want to run the echo \' Bourne command line on the remote host? echo command-string | ssh ... /bin/sh is one solution I gave in my answer, but that means you can't feed data to the stdin of that remote command. – Stéphane Chazelas May 26 '15 at 19:41
  • 1
    Sounds like a more long-lasting solution would be an rexec plugin for ssh, ala the ftp plugin. – Otheus May 26 '15 at 21:37
  • 1
    @myrdd, no it's not, you need either space or tab to separate arguments in a shell command line. If cmd is cmd=(echo "foo bar"), the shell command line passed to ssh should be something like ` 'echo' 'foo bar'. The *first* space (the one before echo) is superflous, but doen't harm. The other one (the ones before 'foo bar') is needed. With '%q', we'd pass a 'echo''foo bar'` command line. – Stéphane Chazelas Dec 4 '18 at 15:18
19

I don't think any implementation of ssh has a native way to pass a command from client to server without involving a shell.

Now, things can get easier if you can tell the remote shell to only run a specific interpreter (like sh, for which we know the expected syntax) and give the code to execute by another mean.

That other mean can be for instance standard input or an environment variable.

When neither can be used, I propose a hacky third solution below.

Using stdin

If you don't need to feed any data to the remote command, that's the easiest solution.

If you know the remote host has an xargs command that supports the -0 option and the command is not too large, you can do:

printf '%s\0' "${cmd[@]}" | ssh user@host 'xargs -0 env --'

That xargs -0 env -- command line is interpreted the same with all those shell families. xargs reads the null-delimited list of arguments on stdin and passes those as arguments to env. That assumes the first argument (the command name) does not contain = characters.

Or you can use sh on the remote host after having quoted each element using sh quoting syntax.

shquote() {
  LC_ALL=C awk -v q=\' '
    BEGIN{
      for (i=1; i<ARGC; i++) {
        gsub(q, q "\\" q q, ARGV[i])
        printf "%s ", q ARGV[i] q
      }
      print ""
    }' "$@"
}
shquote "${cmd[@]}" | ssh user@host sh

Using environment variables

Now, if you do need to feed some data from the client to the remote command's stdin, the above solution won't work.

Some ssh server deployments however allow passing of arbitrary environment variables from the client to the server. For instance, many openssh deployments on Debian based systems allow passing variables whose name starts with LC_.

In those cases you could have a LC_CODE variable for instance containing the shquoted sh code as above and run sh -c 'eval "$LC_CODE"' on the remote host after having told your client to pass that variable (again, that's a command-line that's interpreted the same in every shell):

LC_CODE=$(shquote "${cmd[@]}") ssh -o SendEnv=LC_CODE user@host '
  sh -c '\''eval "$LC_CODE"'\'

Building a command line compatible to all shell families

If none of the options above are acceptable (because you need stdin and sshd doesn't accept any variable, or because you need a generic solution), then you'll have to prepare a command line for the remote host that is compatible with all supported shells.

That is particularly tricky because all those shells (Bourne, csh, rc, es, fish) have their own different syntax, and in particular different quoting mechanisms and some of them have limitations that are hard to work around.

Here is a solution I came up with, I describe it further down:

#! /usr/bin/perl
my $arg, @ssh, $preamble =
q{printf '%.0s' "'\";set x=\! b=\\\\;setenv n "\
";set q=\';printf %.0s "\""'"';q='''';n=``()echo;x=!;b='\'
printf '%.0s' '\'';set b \\\\;set x !;set -x n \n;set q \'
printf '%.0s' '\'' #'"\"'";export n;x=!;b=\\\\;IFS=.;set `echo;echo \.`;n=$1 IFS= q=\'
};

@ssh = ('ssh');
while ($arg = shift @ARGV and $arg ne '--') {
  push @ssh, $arg;
}

if (@ARGV) {
  for (@ARGV) {
    s/'/'\$q\$b\$q\$q'/g;
    s/\n/'\$q'\$n'\$q'/g;
    s/!/'\$x'/g;
    s/\\/'\$b'/g;
    $_ = "\$q'$_'\$q";
  }
  push @ssh, "${preamble}exec sh -c 'IFS=;exec '" . join "' '", @ARGV;
}

exec @ssh;

That's a perl wrapper script around ssh. I call it sexec. You call it like:

sexec [ssh-options] user@host -- cmd and its args

so in your example:

sexec user@host -- "${cmd[@]}"

And the wrapper turns cmd and its args into a command line that all shells end up interpreting as calling cmd with its args (regarless of their content).

Limitations:

  • The preamble and the way the command is quoted means the remote command line ends up being significantly larger which means the limit on the maximum size of a command line will be reached sooner.
  • I've only tested it with: Bourne shell (from heirloom toolchest), dash, bash, zsh, mksh, lksh, yash, ksh93, rc, es, akanga, csh, tcsh, fish as found on a recent Debian system and /bin/sh, /usr/bin/ksh, /bin/csh and /usr/xpg4/bin/sh on Solaris 10.
  • If yash is the remote login shell, you can't pass a command whose arguments contain invalid characters, but that's a limitation in yash that you can't work around anyway.
  • Some shells like csh or bash read some startup files when invoked over ssh. We assume those don't change the behaviour dramatically so that the preamble still works.
  • beside sh, it also assumes the remote system has the printf command.

To understand how it works, you need to know how quoting works in the different shells:

  • Bourne: '...' are strong quotes with no special character in it. "..." are weak quotes where " can be escaped with backslash.
  • csh. Same as Bourne except that " cannot be escaped inside "...". Also a newline character has to be entered prefixed with a backslash. And ! causes problems even inside single quotes.
  • rc. The only quotes are '...' (strong). A single quote within single quotes is entered as '' (like '...''...'). Double quotes or backslashes are not special.
  • es. Same as rc except that outside quotes, backslash can escape a single quote.
  • fish: same as Bourne except that backslash escapes ' inside '...'.

With all those contraints, it's easy to see that one cannot reliably quote command line arguments so that it works with all shells.

Using single quotes as in:

'foo' 'bar'

works in all but:

'echo' 'It'\''s'

would not work in rc.

'echo' 'foo
bar'

would not work in csh.

'echo' 'foo\'

would not work in fish.

However we should be able to work around most of those problems if we manage to store those problematic characters in variables, like backslash in $b, single quote in $q, newline in $n (and ! in $x for csh history expansion) in a shell independant way.

'echo' 'It'$q's'
'echo' 'foo'$b

would work in all shells. That would still not work for newline for csh though. If $n contains newline, in csh, you have to write it as $n:q for it to expand to a newline and that won't work for other shells. So, what we end-up doing instead here is calling sh and have sh expand those $n. That also means having to do two levels of quoting, one for the remote login shell, and one for sh.

The $preamble in that code is the trickiest part. It makes use of the various different quoting rules in all shells to have some sections of the code interpreted by only one of the shells (while it's commented out for the others) each of which just defining those $b, $q, $n, $x variables for their respective shell.

Here's the shell code that would be interpreted by the login shell of the remote user on host for your example:

printf '%.0s' "'\";set x=\! b=\\;setenv n "\
";set q=\';printf %.0s "\""'"';q='''';n=``()echo;x=!;b='\'
printf '%.0s' '\'';set b \\;set x !;set -x n \n;set q \'
printf '%.0s' '\'' #'"\"'";export n;x=!;b=\\;IFS=.;set `echo;echo \.`;n=$1 IFS= q=\'
exec sh -c 'IFS=;exec '$q'printf'$q' '$q'<%s>'$b'n'$q' '$q'arg with $and spaces'$q' '$q''$q' '$q'even'$q'$n'$q'* * *'$q'$n'$q'newlines'$q' '$q'and '$q$b$q$q'single quotes'$q$b$q$q''$q' '$q''$x''$x''$q

That code ends up running the same command when interpreted by any of the supported shells.

  • 1
    The SSH protocol (RFC 4254 §6.5) defines a remote command as a string. It's up to the server to decide how to interpret that string. On Unix systems, the normal interpretation is to pass the string to the user's login shell. For a restricted account, that could be something like rssh or rush that doesn't accept arbitrary commands. There could even be a forced command on the account or on the key that causes the command string sent by the client to be ignored. – Gilles May 25 '15 at 20:54
  • 1
    @Gilles, thanks for the RFC reference. Yes, the assumption for this Q&A is that the login shell of the remote user is usable (as in I can run that remote command I want to run) and one of the main shell families on POSIX systems. I'm not interested in restricted shells or non-shells or force commands or anything that won't allow me run that remote command anyway. – Stéphane Chazelas May 25 '15 at 20:58
  • 1
    An useful reference about the main differences in syntax between some common shells can be found at Hyperpolyglot. – lcd047 May 26 '15 at 4:57
0

tl;dr

ssh USER@HOST -p PORT $(printf "%q" "cmd") $(printf "%q" "arg1") \
    $(printf "%q" "arg2")

For a more elaborate solution, read the comments and inspect the other answer.

description

Well, my solution won't work with non-bash shells. But assuming it's bash on the other end, things get simpler. My idea is to reuse printf "%q" for escaping. Also generally, it's more readable to have a script on the other end, that accepts arguments. But if the command is short, it's probably okay to inline it. Here are some example functions to use in scripts:

local.sh:

#!/usr/bin/env bash
set -eu

ssh_run() {
    local user_host_port=($(echo "$1" | tr '@:' ' '))
    local user=${user_host_port[0]}
    local host=${user_host_port[1]}
    local port=${user_host_port[2]-22}
    shift 1
    local cmd=("$@")
    local a qcmd=()
    for a in ${cmd[@]+"${cmd[@]}"}; do
        qcmd+=("$(printf "%q" "$a")")
    done
    ssh "$user"@"$host" -p "$port" ${qcmd[@]+"${qcmd[@]}"}
}

ssh_cmd() {
    local user_host_port=$1
    local cmd=$2
    shift 2
    local args=("$@")
    ssh_run "$user_host_port" bash -lc "$cmd" - ${args[@]+"${args[@]}"}
}

ssh_run USER@HOST ./remote.sh "1  '  \"  2" '3  '\''  "  4'
ssh_cmd USER@HOST:22 "for a; do echo \"'\$a'\"; done" "1  '  \"  2" '3  '\''  "  4'
ssh_cmd USER@HOST:22 'for a; do echo "$a"; done' '1  "2' "3'  4"

remote.sh:

#!/usr/bin/env bash
set -eu
for a; do
    echo "'$a'"
done

The output:

'1  '  "  2'
'3  '  "  4'
'1  '  "  2'
'3  '  "  4'
1  "2
3'  4

Alternatively, you can do printf's job yourself, if you know what you're doing:

ssh USER@HOST ./1.sh '"1  '\''  \"  2"' '"3  '\''  \"  4"'
  • 1
    That assumes the login shell of the remote user is bash (as bash's printf %q quotes in a bash fashion) and that bash be available on the remote machine. There are also a few problems with missing quotes which would cause problems with whitespace and wildcards. – Stéphane Chazelas Dec 15 '17 at 16:59
  • @StéphaneChazelas Indeed, my solution is probably targeting only bash shells. But hopefully people will find it of use. I tried to address the other issues though. Feel free to tell me if there's anything I'm missing other than the bash thing. – x-yuri Dec 18 '17 at 23:41
  • 1
    Note that it still doesn't work with the sample command in the question (ssh_run user@host "${cmd[@]}"). You still have some missing quotes. – Stéphane Chazelas Dec 19 '17 at 10:10
  • 1
    That's better. Note that the output of bash's printf %q is not safe to use in a different locale (and is also quite buggy; for instance in locales using the BIG5 charset, it (4.3.48) quotes ε as α`!). For that, best is to quote everything and with single quotes only like with the shquote() in my answer. – Stéphane Chazelas Dec 19 '17 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.