11

I've seen couple of similar topics, but they are referring to not quoting variables, which I know could lead to unwanted results.

I saw this code and was wondering would if it be possible to inject something to be run when this line of code executes:

echo run after_bundle

  • I ran into this when I had: target="***LIVE SERVER***"; echo target: $target; and the *** expanded into a folder listing...😬 – Matt Parkins Apr 8 at 11:48
17

For the specific case

echo run after_bundle

quoting is not needed. No quoting is needed because the argument to echo are static strings that contain no variable expansions or command substitutions etc. They are "just two words" (and as Stéphane points out, they are additionally constructed out of the portable character set).

The "danger" comes when you deal with variable data that the shell may expand or interpret. In such cases, care must be taken that the shell does the correct thing and that the result is what's intended.

The following two questions contain relevant information about that:


echo is sometimes used to "protect" potentially harmful commands in answers on this site. For example, I may show how to remove files or move files to a new destination using

echo rm "${name##*/}.txt"

or

echo mv "$name" "/new_dir/$newname"

This would output commands on the terminal instead of actually removing or renaming files. The user could then inspect the commands, decide that they look ok, remove the echo and run again.

Your command echo run after_bundle may be an instruction to the user, or it may be a "commented out" piece of code that is too dangerous to run without knowing the consequences.

Using echo like this, one has to know what the modified command does and one must guarantee that the modified command actually is safe (it would potentially not be if it contained redirections, and using it on a pipeline doesn't work, etc.)

  • Adding quotes isn't sufficient to know what a shell would do, however -- just as you can't tell that echo rm "first file.txt" "second file.txt" is in any way different from echo rm "first" "file.txt" "second" "file.txt", the output from both being the same. If you want to generate a shell command as output, one must use printf '%q ' rm "first file.txt" "second file.txt"; echo or something equivalent that re-generates syntactic quoting that evaluates to the argv passed. – Charles Duffy May 13 '18 at 17:59
  • @CharlesDuffy I really hope nobody copy-paste debugging output and runs it in the shell! – Kusalananda May 13 '18 at 18:14
  • 1
    Generating shell commands and then piping them to sh is not exactly an uncommon pattern, and seeing people ask "why does foo work when I run it on a command line, but this script that emits that exact string with echo in front of the line doesn't?" happens all the time here. More to the point, debugging output isn't helpful if it hides your bugs -- and if your bugs are related to quoting, then echo won't reveal them. – Charles Duffy May 14 '18 at 1:26
27

Just an extra note on top of @Kusalananda's fine answer.

echo run after_bundle

is fine because none of the characters in those 3 arguments¹ passed to echo contain characters that are special to the shell.

And (the extra point I want to make here) there's no system locale where those bytes could translate to characters that are special to the shell.

All those characters are in what POSIX calls the portable character set. Those characters should be present and encoded the same in all character sets on a POSIX system².

So that command line will be interpreted the same regardless of the locale.

Now, if we start using characters outside of that portable character set, it's a good idea to quote them even if they are not special to the shell, because in another locale, the bytes that constitute them may be interpreted as different characters that could become special to the shell. Note that it's whether you're using echo or any other command, the problem is not with echo but with how the shell parses its code.

For instance in a UTF-8:

echo voilà | iconv -f UTF-8 -t //TRANSLIT

That à is encoded as 0xc3 0xa0. Now, if you have that line of code in a shell script and the shell script is invoked by a user who uses a locale whose charset is not UTF-8, those two bytes could make very different characters.

For instance, in a fr_FR.ISO8859-15 locale, a typical French locale using the standard single-byte charset that covers the French language (the same used for most western European languages including English), that 0xc3 byte is interpreted as the à character and 0xa0 as the non-breaking space character.

And on a few systems like NetBSD³, that non-breaking space is considered as a blank character (isblank() on it returns true, it is matched by [[:blank:]]) and shells like bash therefore treat it as a token delimiter in their syntax.

That means that instead of running echo with $'voil\xc3\xa0' as argument, they run it with $'voil\xc3' as argument, which means it won't print voilà correctly.

It gets a lot worse with Chinese character sets like BIG5, BIG5-HKSCS, GB18030, GBK which have many characters whose encoding contains the same encoding as the |, `, \ (to name the worst) (also that ludicrous SJIS, aka Microsoft Kanji, except that it's ¥ instead of \, but still treated as \ by most tools as it's encoded as 0x5c there).

For instance, if in a zh_CN.gb18030 Chinese locale, you write a script like:

echo 詜 reboot

That script will output 詜 reboot in a locale using GB18030 or GBK, 唰 reboot in a locale using BIG5 or BIG5-HKSCS, but in a C locale using ASCII or a locale using ISO8859-15 or UTF-8, will cause reboot to be run because the GB18030 encoding of is 0xd4 0x7c and 0x7c is the encoding of | in ASCII so we end up running:

 echo �| reboot

(that � representing however the 0xd4 byte is rendered in the locale). Example using the less harmful uname instead of reboot:

$ echo $'echo \u8a5c uname' | iconv -t gb18030 > myscript
$ LC_ALL=zh_CN.gb18030 bash ./myscript | sed -n l
\324| uname$
$ LC_ALL=C bash ./myscript | sed -n l
Linux$

(uname was run).

So my advise would be to quote all strings that contain characters outside of the portable character set.

However note that since the encoding of \ and ` are found in the encoding of some of those characters, it's better not to use \ or "..." or $'...' (inside which ` and/or \ are still special), but the '...' instead to quote characters outside the portable character set.

I'm not aware of any system that has a locale where the charset has any character (other than ' itself of course) whose encoding contains the encoding of ', so those '...' should definitely be the safest.

Note that several shells also support a $'\uXXXX' notation to express characters based on their Unicode code point. In shells like zsh and bash, the character is inserted encoded in the locale's charset (though can cause unexpected behaviours if that charset doesn't have that character). That lets you avoid to insert non-ASCII characters in your shell code.

So above:

echo 'voilà' | iconv -f UTF-8 -t //TRANSLIT
echo '詜 reboot'

Or:

echo $'voil\u00e0'
echo $'\u8a5c reboot'

(with the caveat the it could break the script when run in locales that don't have those characters).

Or better, since \ is also special to echo (or at least some echo implementations, at least the Unix compliant ones):

printf '%s\n' 'voilà' | iconv -f UTF-8 -t //TRANSLIT
printf '%s\n' '詜 reboot'

(note that \ is also special in the first argument to printf, so non-ASCII characters are also better avoided there in case they may contain the encoding of \).

Note that you could also do:

'echo' 'voilà' | 'iconv' '-f' 'UTF-8' '-t' '//TRANSLIT'

(that would be overkill but could give you some peace of mind if you're not sure which characters are in the portable character set)

Also make sure never to use the ancient `...` form of command substitution (which introduces another level of backslash processing), but use $(...) instead.


¹ technically, echo is also passed as argument to the echo utility (to tell it how it was invoked), it's the argv[0] and argc is 3, though in most shells nowadays echo is builtin, so that exec() of a /bin/echo file with a list of 3 arguments is simulated by the shell. It's also common to consider the list of arguments as starting with the second one (argv[1] to argv[argc - 1]) as that's the ones the commands mainly act upon.

² a notable exception to that being the ludicrous ja_JP.SJIS locale of FreeBSD systems whose charset has no \ nor ~ character!

³ note that while many systems (FreeBSD, Solaris, not GNU ones though) consider U+00A0 as a [[:blank:]] in UTF-8 locales, few do in other locales like those using ISO8859-15, possibly to avoid this kind of issue.

  • In your first paragraph, you tell us "... of the characters in those 3 arguments passed to echo...", I only count 2 arguments being passed to the command echo, the arguments I can count are run and after_bundle, care to explain how you counted and got to 3 arguments? – Ferrybig May 13 '18 at 11:53
  • 1
    @ViktorFonic, see edit about the number of arguments (and that the main problem is not with echo). See (exec -a foo /bin/echo --help) on a GNU system and with the GNU shell for how to pass an arbitrary first argument to the /bin/echo utility. – Stéphane Chazelas May 13 '18 at 14:09
  • @Ferrybig See Stephane's edit, footnote 1. Arguments to command in the usual C style are array of arguments, with argv[0] being executable name itself. Kinda similar to $0 and positional parameters in shells. – Sergiy Kolodyazhnyy May 13 '18 at 18:51
  • There are 373 encoding in iconv in which ESC is converted to '. Try (as an example) : printf '\x1b'|iconv -f utf8 -t IBM-937|xxd – Isaac May 22 '18 at 23:14
  • There are 173 encoding in which some codepoint (other than ESC) is converted to a '. Try printf '\u2804' | iconv -f utf8 -t BRF | xxd. There are encodings in which there are a lot of codepoints that become '. Around 8695 codepoints in UCS-4 become '. Try printf '\U627' | iconv -cf utf-8 -t UCS-4. Several (37) encoding convert the character 0x127 to a '. Try printf '\U127' | iconv -cf utf8 -t UCS2 |xxd – Isaac May 23 '18 at 0:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.