8

In terminal, if I define some variable char as follows:

export char=\'\\\"\?\!

In effect, char is the string

'\"?!

And then I use the tr command to replace '\"?! with numbers 01234

tr "\'\\\"\?\!" "01234"

And I thought I would get

01234

Instead, I got

0\123

I would be really grateful if someone could explain to me what happened.

It seems replacing each character individually with the sed command avoids this problem, but why?

2
  • Try echo "$char" | tr "\'\\\\\"\?\!" "01234" – fpmurphy Feb 9 '20 at 8:04
  • 2
    Any particular reason you're using export for a shell variable? – Barmar Feb 10 '20 at 9:13
17

Not just the shell, but tr itself also interprets backslash as a special escaping character, see its manual for details. So you need to make sure that tr receives literal \\ (two backslashes) when you want to replace backslashes. This might be done e.g. by char=...\\\\... in the shell, this part doesn't need further explanation since you understand correctly how the shell handles the backslash.

This might be inconvenient for you here, but is convenient in many other situations, and allows sets of characters, or the NUL byte to be part of the search or replace set (which wouldn't be possible otherwise). E.g. to convert NUL-delimited strings to newline-delimited you can do something like tr '\0' '\n' < /proc/1234/environ, or to lowercase a string use tr '[:upper:]' '[:lower:]'. These wouldn't be possible if tr didn't have an escape character.

2
  • Yup. While I typically like to understand every nuance of a command I use, I found it far quicker to often just run "echo" and see if it gives the results I expect. But even then, programs may interpret characters and gobble more backslashes. When dealing with grep, I've often dealt with the four backslashes becomes one situation. I've even encountered situations where eight backslashes become one (perhaps when making script files?). If I used a shorter number of backslashes (2-7), that could be treated as one, but a second backslash would require one more (e.g., 9-15 backslashes become 2). – TOOGAM Feb 9 '20 at 18:19
  • @TOOGAM sounds like you're using the wrong delimiting quotes in those cases you describe. – Ed Morton Feb 9 '20 at 21:01
5

Always enclose strings and scripts in single quotes (') unless you need to use double quotes (") to make the shell interpret it. See https://mywiki.wooledge.org/Quotes. By using double quotes you are inviting the shell in and so putting yourself into "escaping" hell where you have to escape characters within the string for shell to use up first and then escape them again for the tool to use so you need to add multiple layers of escapes instead of 1. Just don't do that, use single quotes instead:

$ printf '%s\n' ''\''\"?!'
'\"?!

$ printf '%s\n' ''\''\"?!' | tr ''\''\\"?!' '01234'
01234

The same is true when you define your variable char. Instead of having all those backslashes:

export char=\'\\\"\?\!

just single-quote the string properly:

$ export char=''\''\"?!'

$ printf '%s\n' "$char"
'\"?!

$ printf '%s\n' "$char" | tr ''\''\\"?!' '01234'
01234

In the above all you need to know is that to get a ' inside a '-enclosed string in shell is '\'' and you need to escape the backslash in the tr so that tr knows to treat it as a literal backslash rather than an escape for the subsequent ".

6
  • 1
    That argument to the first printf could also be written as \''\"?!', i.e. without the empty single-quoted string in the beginning. – ilkkachu Feb 9 '20 at 20:52
  • 2
    While I'm generally sympathetic to the rule of using single-quotes where possible, the need to remember the magic invocation '\'' doesn't really feel like avoiding "escaping hell". – IMSoP Feb 10 '20 at 10:11
  • 1
    @IMSoP, it's not that bad. If we have 'foo'\''bar', the first quote of the '\'' sequence ends the first quoted string ('foo'), the backslash and quote are the escaped quote that we want, and the last quote in the sequence start the quoted string again ('bar'). (Broken to parts, it's 'foo', \', and 'bar') Which is exactly why something like 'foo'\''' is silly in that it just treats the sequence as a "magic" constant. – ilkkachu Feb 10 '20 at 10:34
  • 2
    @IMSoP, oh yes indeed. But here, with a string of all sorts of special characters, including quotes and backslashes... well, it's going to look bad anyway. (unless you have something like Perl's q('\"?!) where you can choose the separator to something you don't need in the string) – ilkkachu Feb 10 '20 at 10:39
  • 1
    @IMSoP When you use single quotes, you don't have to remember how all that works you ONLY have to remember how to include a ', as opposed to if you use double quotes then you have to remember how to include double quotes, escapes, and $ signs. – Ed Morton Feb 10 '20 at 14:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.