2

I find that when writing text as input to another program, any command substitutions in double quotes within the intended text are interpreted and expanded by the shell

The links in the answer here states that single quotes can be used to prevent parameter expansion or command substitution. However I'm finding that enclosing a command substitution in single-quotes also fails to stop the shell from expanding the command substitution

How do you prevent the shell from interpreting command substitutions that are intended as text rather than a command to be executed?

A demonstration

$ echo "`wc -l *`"

attempts to count lines in all files in the current directory

$ echo "'`wc -l *`'"

Same result, i.e. counts lines in all files in the current directory

update From this demonstration I've spotted that the problem seems to be that I am quoting the single quotes. I think enclosing single quotes and ` (backtick) in double quotes preserves the literal meaning of (i.e. suppresses) the single quotes but does not preserve the literal meaning of the backquote (i.e. backtick) that introduces the command substitution.

In my use case the input for another command needs to be quoted. With this document saying that:

A single-quote cannot occur within single quotes

How do you prevent a single-quoted command substitution from being expanded when the single-quoted command substitution is within a (double) quoted string? There should be a way to do it other than using backslash escapes

Actual situation

In a program I'm using the only way to split a description of a task into separate lines is to enclose the description in double-quotes:

$ task add "first line doesn\'t say much
Second line says a lot but part of this line does not appear in the resulting description 'truncate -s0 !(temp_file | temp_dir)' truncates all files to 0 bytes as shown by: '`wc -l *`'"

The resulting description:

first line doesn\ -s0 !(temp_file | temp_dir)' truncates all files to 0 bytes as shown by: 0 file1 10 file2 0 directory1 0 directory2 502 file3 123 file4 162 file5 0 directory3

As you can see

't say much
Second line says a lot but part of this line does not appear in the resulting description 'truncate

is missing from the description and the shell has interpreted 'wc -l *' as a command substitution, thereby including the line counts of all files in the current directory as part of the description

What's causing the shell to remove the part of the argument to task between \ (backslash) and -s, and how do you prevent the shell from interpreting the above single-quoted command substitution (i.e. '`wc -l *`')?

  • 4
    You mean echo '$(ls)' does not produce $(ls) as its output? Can you add the example that is not working for you in the question? – NickD Jul 29 at 20:20
  • Note that adding single quotes within double quotes won't help. – Andy Dalton Jul 29 at 20:42
  • 4
    You know, instead of us guessing, you should put your actual example in the question. – NickD Jul 30 at 0:19
  • 2
    @MyWrathAcademia, like NickD said, please edit your question to show the actual situation, not something where you have removed parts of the scenario. Are you running something like eval in anotherCommand? Or passing the command over SSH? Or just expanding a variable containing the command? Things like that matter. – ilkkachu Jul 30 at 10:19
  • 1
    Note that a shell string is either single quoted or double quoted, whichever statrted the quoting space. There is no such thing as joint quoting (both double and single quotes in effect at the same time). – Isaac Jul 31 at 18:02
2

Here (linebreaks added),

$ task add "first line doesn\'t say much
Second line says a lot but part of this line does not appear in the
resulting description 'truncate -s0 !(temp_file | temp_dir)' truncates
all files to 0 bytes as shown by: '`wc -l *`'"

the whole string is double-quoted, so command substitutions and other expansions will run there. That happens in the shell, before task sees that string, and you'll need to prevent it with backslashes or putting that part in single quotes.

E.g.

$ printf "%s\n" "...shown by: '\`wc -l *\`'"
...shown by: '`wc -l *`'

So,

task add "...shown by: '\`wc -l *\`'"

would pass the string ...shown by: '`wc -l *`' to task. It's up to it what does with that.

If you don't want to use backslashes, here's the way to put it in single quotes:

#               aaaaaaaaaaaaaaaaBBBBBBBBBBBaaa
$ printf "%s\n" "...shown by: '"'`wc -l *`'"'"
...shown by: '`wc -l *`'

(The a's mark the double-quoted parts, the B's the single-quoted parts. They are just concatenated on the shell command line. The literal single quotes are within the double-quoted strings.)


As for the single quote and the backslash, you don't need to escape a single quote within double quotes, and in fact the backslash will remain there:

$ printf "%s\n" "foo'bar"
foo'bar
$ printf "%s\n" "foo\'bar"
foo\'bar

From what you show, it seems like task removes at least the first single-quoted string from the argument (plus a word after that, since the removed part was 't say much ... 'truncate)

The shell will not do that, this works fine:

$ printf "%s\n" "a 'quoted string' to test"
a 'quoted string' to test

What's causing the shell to remove the part of the argument to task between \ (backslash) and -s,

It's highly likely it's not the shell doing that.

and how do you prevent the shell from interpreting the above single-quoted command substitution (i.e. '`wc -l *`')?

It's not single-quoted, it's double-quoted with quoted single quotes next to it.

  • I just got the chance to read and test your answer. It works finely; using string concatenation to put the part whose literal meaning is not preserved by double-quotes inside single-quotes is very useful. Although I noticed unexpected behavior in the string concatenation: I assume an implicit + (plus) operator is between the strings being combined so why is it that when there is no space between strings the new string created from the combined strings has no space where they were combined but if you separate the strings with a space then the new string has a space where they were combined? – MyWrathAcademia Aug 5 at 19:08
  • @MyWrathAcademia, hmm, separate with a space how? If you write something like somecmd "foo" 'bar', then there's actually two arguments (in the same way as writing ls -l foo.txt passes the option and the filename as separate arguments to ls.) Though if you happen to test with echo, you won't see it, since it combines the arguments with a single space... – ilkkachu Aug 5 at 19:10
  • What I mean is, task add "...shown by:'"'`wc -l *`'"'" creates the string "...shown by:'`wc -l *`' (i.e. no white space character in between characters where the strings were combined) where as task add "...shown by:'" '`wc -l *`' "'" creates the string "...shown by:' `wc -l *` ' (i.e. white space character in between the characters where the strings were combined) – MyWrathAcademia Aug 5 at 19:35
  • @MyWrathAcademia, yes, that's exactly what I mean, the latter doesn't create one string, it creates three. If the command itself then concatenates them with a space, well, that's another thing. But try with something that does otherwise: printf ":%s:\n" "...shown by:'" '`wc -l *`' "'" vs. the one without spaces. (retake, I messed up the backticks) – ilkkachu Aug 5 at 19:47
11

Use single-quote strong quoting:

printf '%s\n' '`wc -l *`'

And if you want to also include single quotes in that argument passed to printf, you'd need to use different quotes for ' itself like:

printf '%s\n' '`wc -l *` and a '"'"' character'

Or:

printf '%s\n' '`wc -l *` and a '\'' character'

Other alternatives include escaping the ` with backslash inside double quotes:

printf '%s\n' "\`wc -l *\` and a ' character"

Or have ` be the result of some expansion:

backtick='`'
printf '%s\n' "${backtick}wc -l *${backtick} and a ' character"

Also note:

cat << 'EOF'
`wc -l *` and a ' character and a " character
EOF

to output arbitrary text without having to worry about quoting (note the quotes around the first EOF).

You can also do:

var=$(cat << 'EOF'
echo '`wc -l *`'
EOF
)

Which with ksh93 or mksh you can optimise to:

var=$(<<'EOF'
echo '`wc -l *`'
EOF
)

(also works in zsh, but still runs cat in a subshell there) for $var to contain literally echo '`wc -l *`'.

In the fish shell, you can embed ' within '...' with \':

printf '%s\n' '`wc -l *` and a \' character'

but anyway ` is not special there, so:

printf '%s\n' "`wc -l *` and a ' character"

would work as well.

In rc, es or zsh -o rcquotes, you can insert a ' within '...' with '':

printf '%s\n' '`wc -l *` and a '' character'

See How to use a special character as a normal one? for more details.

  • In your first example that includes a single-quote in printf's argument why are you single-quoting the single quote that you want it's literal meaning to be preserved when you already double quoted it? Similarly in your second example that includes a single-quote in the argument for printf why are you single-quoting a single-quote that has already been escaped using a backslash? Isn't the single-quote in your examples that I highlighted above redundant? – MyWrathAcademia Jul 30 at 19:38
  • Your trick to prevent the shell from interpreting special characters in a here document is very useful. And your solution to store the result of the command substitution in a variable is also immensely useful. Can you explain why the value of $var is '`wc -l *`' where as $ echo $var outputs echo '`wc -l *`'? Overall this answer is a wealth of wisdom – MyWrathAcademia Jul 30 at 20:05
  • I'm not really familiar with variable expansion, why not printf '%s\n' "$backtick wc -l * $backtick and a ' character"? Does ${backtick} prepend text when in front of a string and append text when behind a string? In other words, can you clarify how ${backtick} works? I'm pleasantly surprised and intrigued that variable expansion (if that's the right word) prevents the shell from interpreting special characters. Does this apply to all special characters not just the backtick? May be using variable expansion to prevent interpretation of special characters by the shell is exactly what I need – MyWrathAcademia Jul 30 at 20:11
  • Please edit your answer to include the actual situation that I added to my question because the variable expansion you suggested is not working – MyWrathAcademia Jul 30 at 21:39
  • 1
    @Wildcard Operator precedence imply that both operators are used (executed), like 2+3*4 becomes 14, both sum and multiply are executed. In this case, the expansion of $ only happens once. That is: it shaves one level of $ expansion. That is: a var expansion doesn't happen after a com expansion (try echo $(echo '$a')) nor a com expansion will happen after a var expansion (you posted it already). That is not true inside an arithmetic expansion, try a=23 b=a c=b d=c e=$((d)); echo "a=$a b=$b c=$c d=$d e=$e" it keeps expanding vars as long as there is a valid var name to expand. – Isaac Jul 31 at 17:50
1

You can escape the backticks by using a backslash as shown below:

echo "\`wc -l *\`"
  • I mentioned prefering solving the problem without using backticks because you would need to escape every character you don't want to be interpreted by the shell where as if quoting could solve the problem you could use one single-quote to quote a substring that contains special characters such as $, `` ` `` and ` \ ` that you don't want to be interpreted by the shell – MyWrathAcademia Jul 29 at 21:23
  • ok, so the reason you need to use double quotes is because you have a variable that needs to be interpreted as well? So, as a result, echo 'wc -l *' wont work as solution? – user138278 Jul 29 at 22:02
0

To escape any arbitrary character sequence so that no special characters are processed and no substitution is done:

  1. Replace every ' with '\''
  2. Enclose the resulting string in single quotes.

For example:

`wc -l 'my file.txt'`

Becomes

'`wc -l '\''my file.txt'\''`'

Note if the string starts or ends with ', you end up with '' at the start/end, which does nothing and can be removed but it's valid to leave it there (e.g. to keep a string-escaping function simple).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.