12

I have browser-based shell/terminal that executes bash commands and I'm escaping spaces but it turns out that parenthesis also need to be escaped. What other characters need to be escaped for file names that are not in quotes?

  • It is not at all clear what you are trying to do. And what has autocompletion got to do with anything? – Michael Vehrs Feb 24 '17 at 16:25
  • @MichaelVehrs I have command mv filename filename-foo in bash, which characters do I need to escape, besides spaces and parenthesis? – jcubic Feb 24 '17 at 16:50
  • @MichaelVehrs removed autocomplete mention. – jcubic Feb 24 '17 at 16:53
  • Basically, all shell metacharacters, such as whitespace, shell globs, history events. For this reason, you should quote the whole filename, rather than escaping individual characters: mv "$filename" "$newname". – Michael Vehrs Feb 27 '17 at 6:57
  • @MichaelVehrs I know that quoting will solve the issue with escaping but I need to use filename without quotes. I have terminal emulator and it should work when user don't use quotes. – jcubic Feb 27 '17 at 8:28
21

The simple solution is to put a single quote (') at the beginning and another single quote at the end, and to replace every ' character inside the file name by the 4-character sequence '\''. All characters lose their special meaning inside a single-quoted string, except ' itself which marks the end of the string. The sequence '\'' ends the single-quoted literal, immediately follows by a quoted single quote, and opens a new single-quoted literal. Thus the file name

This file's name has some weird characters!
Will you manage to escape them?

can be quoted as follows:

somecommand 'This file'\''s name has some weird characters!
Will you manage to escape them?'

Double quotes have more complex escaping rules and don't allow you to include an exclamation mark ! if history substitution is activated, so I won't consider them further.

An alternative approach is to protect characters with backslashes. This works for every character except newlines; for a newline, putting it inside single quotes (or double quotes) is the only solution. If you want to minimize the number of backslashes to present the quoted name to the user, you may restrict it to places where the backslash is needed; however, the more backslashes you omit, the more you risk forgetting one that's needed. Letters, digits and non-ASCII characters are always ok¹. Quote whitespace and punctuation whenever you're unsure.

With a typical shell (ksh, bash or zsh), you need to quote the following characters in at least some circumstances.

  • Whitespace (space, tab, newline — remembering that newlines can't be quoted with a backslash).
  • ! — history expansion.
  • " — shell syntax.
  • # — comment start when preceded by whitespace; zsh wildcards.
  • $ — shell syntax.
  • & — shell syntax.
  • ' — shell syntax.
  • ( — even in the middle of a word: ksh extended globs (also available in bash and zsh); zsh wildcards.
  • ) (see ()
  • * — sh wildcard.
  • , — only inside brace expansion.
  • ; — shell syntax.
  • < — shell syntax.
  • = — in zsh, when it's at the beginning of a file name (filename expansion with PATH lookup).
  • > — shell syntax.
  • ? — sh wildcard.
  • [ — sh wildcard.
  • \ — shell syntax.
  • ] — you may get away with leaving it unquoted.
  • ^ — history expansion; zsh wildcard.
  • ` — shell syntax.
  • { — brace expansion.
  • | — shell syntax.
  • } — needs to be escaped in zsh, other shells are more lenient when there's no matching opening brace.
  • ~ — home directory expansion when it's at the beginning of a file name; zsh wildcard; always safe when it's the last character.

A few more characters can require special handling sometimes:

  • - isn't special for the shell, but when it's at the beginning of a command argument, it indicates an option. It can't be protected with quotes since the special handling is in the command, not in the shell. To protect a file name that begins with -, you can put ./ before it — this way it's still the same file, but the argument doesn't begin with - anymore.
  • . isn't special in itself, but dot files are excluded from * globs by default.
  • : isn't special for the shell, but some commands parse it specially, e.g. to indicate a remote file (hostname:filename). Consult the documentation of the command to see how to cope with file nams containing colons.

¹ Unless the user has configured alternate history expansion characters. Some shells allow this. This is another reason to use single quotes rather than backslashes.

  • Newlines, at least LF only, can be quoted with a backslash in some shells. – iBug Apr 9 '17 at 17:42
  • @iBug No. In any sh-like shell, backslash+newline expands to an empty string, not to a newline. – Gilles 'SO- stop being evil' Apr 9 '17 at 18:16
  • This is a good answer, thank you. Is there a link to where this information is 'officially' available? – slashmais Aug 15 '17 at 8:24
  • @slashmais What information? The syntax of the shell language is “officially” available in various manuals and specifications, but going from there to finding out exactly which characters need quoting where isn't completely straightforward. – Gilles 'SO- stop being evil' Aug 15 '17 at 16:43
  • 1
    I also found this: tecmint.com/manage-linux-filenames-with-special-characters - it is not as compact as your list, but has plenty examples. (by-the-by: "What information?" is not a cool response when the topic here concerns the characters) – slashmais Aug 15 '17 at 17:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.