5

Is there any consistent logic to it?

some-command "${somevariable//some pattern/'how does this get parsed?'}"

I've posted some conclusions and raw tests below as an "answer" but they're not a full answer by any means. The Bash man page appears silent on the subject.

5

As discussed in the comments, this seems to have changed between versions of Bash. I think this is the relevant change in bash-4.3-alpha (changelog):

zz. When using the pattern substitution word expansion, bash now runs the replacement string through quote removal, since it allows quotes in that string to act as escape characters. This is not backwards compatible, so it can be disabled by setting the bash compatibility mode to 4.2.

And the description for shopt -s compat42 (online manual):

compat42
If set, bash does not process the replacement string in the pattern substitution word expansion using quote removal.

The quoting single-quotes example:

$ s=abc\'def; echo "'${s//\'/\'\\\'\'}'"
'abc'\''def'

$ shopt -s compat42
$ s=abc\'def; echo "'${s//\'/\'\\\'\'}'"
'abc\'\\'\'def'

$ bash --version | head -1
GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)

Workaround: put the replacement string in a variable, and don't use quotes inside the replacement:

$ shopt -s compat42
$ qq="'\''"; s=abc\'def; echo "'${s//\'/$qq}'";
'abc'\''def'
$ qq="'\''"; s=abc\'def; echo "'${s//\'/"$qq"}'";
'abc"'\''"def'

The funny thing is, that if the expansion is unquoted, then the quotes are removed after the substitution, in all versions. That is s=abc; echo ${s/b/""} prints ac. This of course doesn't happen with other expansions, e.g. s='a""c' ; echo ${s%x} outputs a""c.

1

General rules by reverse engineering:

  • Quotes must be coupled (completed)
  • Quotes are preserved (included in the actual replacement)
  • Backslashes are preserved if they come before an arbitrary letter
  • Backslashes are preserved if they escape a single quote
  • A backslash backslash sequence is reduced to one backslash even within single quotes
  • You can't escape a single quote within single quotes
  • Parameter expansion works inside single quotes the same as outside
  • If a dollar sign is escaped with a backslash the dollar sign is preserved literally and the backslash is removed

And a conclusion:

  • There is absolutely no way to produce the literal sequence '\'' as a substitution through parameter expansion.
  • However, it is very easy to produce the literal sequence "'\''" as a substitution.

Some raw tests follow.

[vagrant@localhost ~]$ echo "$0"
-bash
[vagrant@localhost ~]$ echo "${0//a/x}"
-bxsh
[vagrant@localhost ~]$ echo "${0//a/some long string  with spaces}"
-bsome long string  with spacessh
[vagrant@localhost ~]$ echo "${0//a/"quoted string"}"
-b"quoted string"sh
[vagrant@localhost ~]$ echo "${0//a/"unfinished quote}"
> wat
> }"
-b"unfinished quote}"
wat
sh
[vagrant@localhost ~]$ echo "${0//a/\"escaped quote}"
-b"escaped quotesh
[vagrant@localhost ~]$ echo "${0//a/\\escaped escape}"
-b\escaped escapesh
[vagrant@localhost ~]$ echo "${0//a/\'escaped single quote}"
-b\'escaped single quotesh
[vagrant@localhost ~]$ echo "${0//a/''}"
-b''sh
[vagrant@localhost ~]$ echo "${0//a/''''}"
-b''''sh
[vagrant@localhost ~]$ echo "${0//a/'''}"
> a'b}c"d
-b'''}"
a'bshcd
[vagrant@localhost ~]$ echo "${0//a/'''}"
> w'x}y"z
-b'''}"
w'xshyz
[vagrant@localhost ~]$ echo "${0//a/'\'\\"a test'\'}"
> ^C
[vagrant@localhost ~]$ echo "${0//a/'\''\\"a test'\'}"
-b'\''\"a test'\'sh
[vagrant@localhost ~]$ echo "${0//a/'\''\\"a test'\$0'}"
> ^C
[vagrant@localhost ~]$ echo "${0//a/\\"a test'\$0'}"
> w}x"y
-b\"a test'$0'}"
wshxy
[vagrant@localhost ~]$ echo "${0//a/\\\"a test'\$0'}"
-b\"a test'$0'sh
[vagrant@localhost ~]$ echo "${0//a/\\\"a test'\\'$0'}"
> ^C
[vagrant@localhost ~]$ echo "${0//a/\\\"a test'\\$0'}"
-b\"a test'\-bash'sh
[vagrant@localhost ~]$ 
  • I don't have time for a full answer right now, but here is a tip: Instead of testing echo "${0//bar/qux}", test temp=${0//bar/qux}; echo "$temp" (notice lack of double quotes in assignment to temp). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignment q=${...} (but not in local q=${...} or export q=${...}) expansions happen as if quoted, so you can work with the substring replacement by itself. – mtraceur Dec 15 '17 at 8:13
  • But also here is a minimal example of producing the literal '\'' as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes: q=abc\'def; echo "$q"; echo "${q//\'/\'\\\'\'}". And another minimal example using double-quotes-in-double-quotes: q=abc\'def; echo "$q"; echo "${q//"'"/"'\''"}". Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents: q=abc\'def; echo "$q"; q=${q//\'/\'\\\'\'}; echo "$q" and q=abc\'def; echo "$q"; q=${q//"'"/"'\''"}; echo "$q" – mtraceur Dec 15 '17 at 8:23
  • @mtraceur, what version of what shell are you using? With bash 4.1.2 your first version produces abc\'\\'\'def – Wildcard Dec 15 '17 at 8:31
  • 1
    But here's one that works in both 4.4 and 3.2: qq="'\''"; s=abc\'def; echo "$s"; echo "'${s//\'/$qq}'"; (i.e. put the replacement string in variable) – ilkkachu Dec 15 '17 at 9:50
  • 1
    ksh seems to work like the newer Bash, but my zsh doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me. – ilkkachu Dec 15 '17 at 10:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.