15

This question is not about how to write a properly escaped string literal. I couldn't find any related question that isn't about how to escape variables for direct consumption within a script or by other programs.

My goal is to enable a script to generate other scripts. This is because the tasks in the generated scripts will run anywhere from 0 to n times on another machine, and the data from which they are generated may change before they're run (again), so doing the operations directly, over a network will not work.

Given a known variable that may contain special characters such as single quotes, I need to write that out as a fully escaped string literal, e.g. a variable foo containing bar'baz should appear in the generated script as:

qux='bar'\''baz'

which would be written by appending "qux=$foo_esc" to the other lines of script. I did it using Perl like this:

foo_esc="'`perl -pe 's/('\'')/\\1\\\\\\1\\1/g' <<<"$foo"`'"

but this seems like overkill.

I have had no success in doing it with bash alone. I have tried many variations of these:

foo_esc="'${file//\'/\'\\\'\'}'"
foo_esc="'${file//\'/'\\''}'"

but either extra slashes appear in the output (when I do echo "$foo"), or they cause a syntax error (expecting further input if done from the shell).

  • alias and/or set fairly universally – mikeserv Jul 28 '18 at 7:48
  • @mikeserv Sorry, I'm not sure what you mean. – Walf Jul 28 '18 at 14:39
  • alias "varname=$varname" varname or var=value set – mikeserv Jul 28 '18 at 14:58
  • 2
    @mikeserv That's not enough context for me to understand what your suggestion is supposed to do, nor how I would use it as a generic method of escaping any variable. It's a solved problem, mate. – Walf Aug 1 '18 at 5:58
22

Bash has a parameter expansion option for exactly this case:

${parameter@Q} The expansion is a string that is the value of parameter quoted in a format that can be reused as input.

So in this case:

foo_esc="${foo@Q}"

This is supported in Bash 4.4 and up. There are several options for other forms of expansion as well, and for specifically generating complete assignment statements (@A).

  • 6
    Neat, but only have 4.2 which gives bad substitution. – Walf Jul 18 '17 at 6:26
  • 2
    The Z shell equivalent is "${foo:q}". – JdeBP Jul 18 '17 at 12:04
  • Really save my life! "${foo@Q}" works! – hao Jun 3 at 6:07
  • @JdeBP that Z shell equivalent doesn't work. Any other ideas for zsh? – Steven Shaw Jun 12 at 5:05
  • I found the answer: "${(@qq)foo}" – Steven Shaw Jun 12 at 11:23
10

Bash provides a printf builtin with %q format specifier, which performs shell escaping for you, even in older (<4.0) versions of Bash:

printf '[%q]\n' "Ne'er do well"
# Prints [Ne\'er\ do\ well]

printf '[%q]\n' 'Sneaky injection $( whoami ) `ls /root`'
# Prints [Sneaky\ injection\ \$\(\ whoami\ \)\ \`ls\ /root\`]

This trick can also be used to return arrays of data from a function:

function getData()
{
  printf '%q ' "He'll say hi" 'or `whoami`' 'and then $( byebye )'
}

declare -a DATA="( $( getData ) )"
printf 'DATA: [%q]\n' "${DATA[@]}"
# Prints:
# DATA: [He\'ll\ say\ hi]
# DATA: [or\ \`whoami\`]
# DATA: [and\ then\ \$\(\ byebye\ \)]

Note that the Bash printf builtin is different than the printf utility which comes bundled with most Unix-like operating systems. If, for some reason, the printf command invokes the utility instead of the builtin, you can always execute builtin printf instead.

  • I'm not sure how that helps if what I'd need printed would be 'Ne'\''er do well', etc., i.e. quotes included in the output. – Walf Jul 30 '18 at 3:15
  • 1
    @Walf I think you're not understanding that the two forms are equivalent, and both are perfectly as safe as each other. E.g. [[ 'Ne'\''er do well' == Ne\'er\ do\ well ]] && echo 'equivalent!' will echo equivalent! – Dejay Clayton Jul 30 '18 at 17:39
  • I did miss that :P however I prefer the quoted form as it's easier to read in a syntax-highlighting viewer/editor. – Walf Jul 31 '18 at 23:43
  • @Walf it seems like your approach is pretty dangerous, considering that in your example Perl, passing a value like 'hello' results in the incorrect value ''\''hello'', which has an unnecessary leading empty string (the first two single quotes), and an inappropriate trailing single quote. – Dejay Clayton Aug 1 '18 at 3:02
  • 1
    @Walf, for clarification, $'escape-these-chars' is the ANSI-C quoting feature of Bash that causes all characters within the specified string to be escaped. Thus, to easily create a string literal that contains a newline within the filename (e.g. $'first-line\nsecond-line'), use \n within this construct. – Dejay Clayton Aug 6 '18 at 19:56
5

I guess I didn't RTFM. It can be done like so:

q_mid=\'\\\'\'
foo_esc="'${foo//\'/$q_mid}'"

Then echo "$foo_esc" gives the expected 'bar'\''baz'


How I'm actually using it is with a function:

function esc_var {
    local mid_q=\'\\\'\'
    printf '%s' "'${1//\'/$mid_q}'"
}

...

foo_esc="`esc_var "$foo"`"
  • 2
    I would use @michael-homer's solution if my version supported it. – Walf Jul 26 '17 at 1:36
1

There are several solutions to quote a var value:

  1. alias
    In most shells (where alias is available)(except csh, tcsh and probably others csh like):

    $ alias qux=bar\'baz
    $ alias qux
    qux='bar'\''baz'
    

    Yes, this works in many sh-like shells like dash or ash.

  2. set
    Also in most shells (again, not csh):

    $ qux=bar\'baz
    $ set | grep '^qux='
    qux='bar'\''baz'
    
  3. typeset
    In some shells (ksh, bash and zsh at least):

    $ qux=bar\'baz
    $ typeset -p qux
    typeset qux='bar'\''baz'             # this is zsh, quoting style may
                                         # be different for other shells.
    
  4. export
    First do:

    export qux=bar\'baz
    

    Then use:
    export -p | grep 'qux=' export -p | grep 'qux='
    export -p qux

  5. quote
    echo "${qux@Q}"
    echo "${(qq)qux}" # from one to four q's may be used.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.