POSIX Shell: inside of double-quotes, are there cases where `\` fails to escape `$`, ```, `"`, `\` or `<newline>`?

Question

Per the POSIX Shell Command Language Page:

\

The <backslash> shall retain its special meaning as an escape character (see Escape Character (Backslash)) only when followed by one of the following characters when considered special:

$ ` " \ <newline>

This would seem to imply that escaping these five characters with a backslash would not have the effect of escaping them and having them be treated literally if they are not "special".

Am I interpreting this correctly, and if so, are there cases where escaping one of these five special characters with a \ would not have the intended effect of escaping it?

Answer 1

@MichaelHomer explains it very well. Let's try a few practical cases with PS1='\$ ':

$ echo "$ at start"
$ at start
$ echo "at end $"
at end $
$ echo "$before"

$ echo "after$"
after$

So $ is only "special" before a word, making it a parameter substitution. What happens if we put a backslash before all of them?

$ echo "\$ at start"
$ at start
$ echo "at end \$"
at end $
$ echo "\$before"
$before
$ echo "after\$"
after$

Only the "special" line changes - the dollar sign is now considered literal. It was always considered literal. What happens with other characters?

$ echo "\ at start"
\ at start
$ echo "at end \"
> ^C
$ echo "\before"
\before
$ echo "after\"
> ^C

So backslash is just another literal character before a non-special character. (^C is where I had to cancel the command line because the quote character had been escaped.)

Answer 2

A character, say $, has a special meaning and is considered special when it has some impact other than being a literal part of data. In $x, $ has a special meaning introducing parameter expansion.

What this passage is saying is that the backslash has its escape-character meaning only when it precedes a one of these characters with a special meaning. Ordinarily, the backslash has broader escaping meaning:

A <backslash> that is not quoted shall preserve the literal value of the following character, with the exception of a <newline>

For example, "\$x" is the string "$x", while "a\ b" is the string "a\ b", because the backslash in the second case precedes an unlisted character and so has no special meaning. Outside double quotes, however, a\ b is the string "a b", with the backslash causing the space to be included literally, and the backslash itself removed.

The passage you quoted isn't saying that sometimes the characters keep their special meaning despite being escaped, but rather that the backslash has no special meaning unless it would be impacting one of them.

---

There is a potential dangling modifier here: are the characters considered special, or the backslash?

There are some potential cases where some of those characters inside double quotes may not have a special meaning - most obviously within embedded expansions described in the immediately-preceding points to the one you quoted, but also in the delimiter of a here-document. Embedded expansions have their own quoting rules, however, as do here-document delimiters, so it's not clear to me that it can practically arise. If it's the backslash, all it says is that a double backslash doesn't escape anything.

The best case I have for where, if it is limiting the characters, it might come up is here-document delimiters:

cat <<"A$B"
A$B
cat <<"A\$B"
A$B

Both of those do the same thing on all my shells. Arguably the $ is not special, since it was included in the delimiter the first time, and so the \ inside double quotes should not have escaped it the second time and should have been literally included. I don't think it's clear that this is required behaviour but it does seem consistent with the letter of the law.

For embedded expansions they either disavow any effect from the quotes (both kinds of $(...)) or have both their own quoting rules and no unique way to have something be quoted (all kinds of ${...} and ` ... `), and I haven't managed to construct something that clearly should have a non-special version of one of those characters and a non-special backslash before it.

I suspect the bolded text is currently redundant. The potential dangling modifier at the least makes it ambiguous, and perhaps this is a phrasing bug to file for the errata or next update.

Answer 3

Lets try to simplify.
The text, as written is:

The <backslash> shall retain its special meaning as an escape character (see Escape Character (Backslash)) only when followed by one of the following characters when considered special:

We can convert some parts of it:

1. The <backslash> ==> The B

2. ... shall retain its special meaning as an escape character
   ... \__________/------------------------\__________________/
   ==> still is -------------------------------- an escape character
   ==> still is an escape

   As it was in un-quoted strings.

3. ... only

4. ... when followed by one of the following characters ==> A B before [$`"\NL]

5. ... when considered special

Then, 1,2,and 4 read as: B still is an escape before [$`"\NL]

• The controversial part is: when considered special

  What is being considered special ?

  I believe that it is >> "the characters" << . As written:

  one of the following characters when considered special:

  Q: For example: when is a " special (inside double quotes)?

  A: Always (not considering nested structures like echo "$(sed 's/["]//' file)").

  So, a backslash will always quote a ": echo "a\"b\"b\"" (again: no nested structures).

• Then, 1,2,3,4 and 5 should read as:

  B still is an escape only before special [$`"\NL]

Then, the answer to your question:

are there cases where \ fails to escape $, ```, ", \ or <newline>?

Is: YES, when the characters [$`"\NL] are not special.

They (the characters) may be not special inside nested structures (for example) yet inside quotes.