Null and escape characters

I know that a filename in Linux has no restriction whatsoever except for two characters '/' and '\0'. I know that '/' is prohibited because it is a directory separator but is there any other reason ?

Also on my terminal I can create a file or a directory with the name \0. So I wonder how to write the null character correctly because obviously it shouldn't allow me to have a filename with null in it

mkdir '\0' will create a directory named \0

One more question, If I want to include $ in my filename, I can use the backslash mkdir \$myfile will create a directory named $myfile However, I can do the same if I surround the dollar sign with single quotes and double quotes mkdir \$myfile is the same as mkdir '$'myfile is the same as mkdir "$"myfile is the same as mkdir '$myfile' is the same as mkdir "$myfile"

So my question is, Are the single and double quotes a substitution for the escape backslash character?

Also what other characters need escaping in bash besides $, (space) and backslash ? • mkdir '\0' creates a two-character file; this is not the NUL character you were looking for ('\0' is from C). – thrig Aug 17 '15 at 23:37 • Those are two different and unrelated questions. You should split that into two questions on this site. The moderators can help you do that. I've flagged it to their attention. – Stéphane Chazelas Aug 18 '15 at 7:36 4 Answers Printing the null character On many recent shells you can write null character with dollar single quotes format $'\0', hexadecimal format \x00, unicode format \u0000 or \U00000000, or just as you tried with octal: '\0'. The point is that the command has to understand what to do with backslash-escaped characters. For example in case of echo usually one needs to add -e option and in case of printf that would be %b.

Let's check if it works:

$echo -ne '\0'$


So produces nothing, just like echo -ne '', similar

$printf '%b' '\0'$


Let's add some characters around (I will stick with printf '%b' from now on as more robust, but similar effect is with echo -ne):

$printf '%b' a'\0'b ab  Only two characters were printed, where did the null go? $ printf '%b' a'\0'b | wc -c
3


Let's compare it with a''b:

$printf '%b' a''b | wc -c 2  Last more check that we really print null character before trying to create file, let's pass the printed value to the command which will throw the error, like xargs: $ printf '%b' a'\0'b | xargs echo
xargs: Warning: a NUL character occurred in the input.  It cannot be
passed through in the argument list.  Did you mean to use the --null option?
a


Notice how only a was printed at the end. Of course xargs -0 works fine:

$printf '%b' a'\0'b | xargs -0 echo a b  Creating the file with null? Now let's try to create file with null character: $ touch $'\0' touch: cannot touch ‘’: No such file or directory$ mkdir $'\0' mkdir: cannot create directory ‘’: No such file or directory # let's try another approach - using printf in command substitution:$ touch "$(printf '%b' '\0')" touch: cannot touch ‘’: No such file or directory$ mkdir "$(printf '%b' '\0')" mkdir: cannot create directory ‘’: No such file or directory  The result is exactly the same as in touch '', it seems null is just ignored all together. What if we skip double quotes around command substitution? $ touch $(printf '%b' '\0') touch: missing file operand Try 'touch --help' for more information.$ mkdir $(printf '%b' '\0') mkdir: missing operand Try 'mkdir --help' for more information.  This is the same situation as touch/mkdir without arguments at all. Yet another result is if we surround null with text: $ touch "$(printf '%b' a'\0'b)"$ ls
a   # in zsh
ab  # in bash


One can also try to redirect standard output to $'\0' but all one gets is different kind of error. • Using an hex viewer (like xxd) really helps. For example: printf '%b' a'\0'b|xxd. Of course, the simpler and almost always present od could also do the job: printf '%b' a'\0'b|od -An -tx1 – user79743 Aug 18 '15 at 4:49 • The problem in bash with touch "$(printf '%b' a'\0'b)" is the 'command substitution'. This echo "$(printf '%b' a'\0'b)"|xxd prints 6162, the null is removed by the 'command substitution'. Instead, this: printf '%b' a'\0'b|xxd does print the null: 610062. Instead zsh acts as 'c code', an string ends where a null exists. – user79743 Aug 18 '15 at 4:57 • echo -ne '\0' outputs a 0 byte in all the echo implementations that support that syntax. It's just that the NUL character, for a terminal, is a control character, the control character that does nothing (for instance it's used sometimes for slowing down the flow of character on a serial line without flow control to let the terminal (like tele-typewriters in the olden days) do its things like return the carriage). So it's just that the NUL character is not visible in a terminal (and it's not the only one). – Stéphane Chazelas Aug 18 '15 at 8:58 • I think you will have better results with printf '%s\0%s' a b. To just print the null character, use printf '\0'. – ErikE Jul 13 at 6:46 Single/double quotes vs. backslash: single quotes and backslashes are equivalent in quoting power. It's a lot more convenient to use single quotes to quote a long string with spaces, tabs, newlines, ()[]*$><?|{}~&;"^!# and probably other characters I'm forgetting. But you could achieve exactly equivalent results with just backslashes (beware of the overloading of backslashes within backticks (...) though)

Double quotes are unique, though. $ expands inside double-quotes, but not single. "$foo" expands foo, but protects the expanded result from word-splitting and glob expansion.

http://mywiki.wooledge.org/BashFAQ might be a good place to start. The bash manual doesn't spend much time on how to use all the features it describes, just how they work individually.

It's literally not possible to pass a string containing a zero-byte as a command line argument, or to a system call. The ABI (application binary interface) that specifies exactly how data is passed between processes and the kernel uses C strings for everything (except binary data), including command line arguments and file / path arguments to system calls. C strings are character arrays where the end-of-string is marked by a zero-byte. There is no way to "escape" a zero byte to indicate that it's not the end of a string.

Any attempt to do something like touch $'foo\0bar' would just result in touch seeing its argument list as argv[0] = "/bin/touch"; argv[1] = "foo";  Even though sitting in memory, argv[1] = "foo\0bar\0", the first \0 marks the end of the string. Actually, "foo\0bar\0" wouldn't make it as far as the new process's argv. It wouldn't make it out of the argv array in the exevce(2) system call that ran touch. And even if you wrote a C or perl program with character arrays / strings containing null bytes, passing them to a system call like open(2) would cause the same interpretation of the string by kernel. System calls that need to handle arbitrary data, like read(2) and write(2), take a length argument as well as a pointer to the buffer. It's not even possible to do much of anything with null bytes with bash. As jimmij points out, the bash syntax for writing a string literal with escape-sequence processing is $'string', but writing a \0 in your string literal acts as a string terminator inside bash. I guess this means bash stores strings internally as C strings, not with an explicit length.

str=$'foo\0bar' echo "${#str}"   # 3, showing that bash isn't even storing it in a variable.
echo "$str" | wc -c # 4. wouldn't work even if${#str} did: echo's cmdline would eat it
wc -c <<< $'foo\0bar' # 4 (includes a newline)  So we can't use this syntax to send a null byte anywhere. We'd have to use tr or something. However, bash printf does have some limited support for \0. The %b conversion processes backslash escapes, including \0. And printf already processes such escapes in its format string. • printf '\0' prints a zero byte. Pipe it into hexdump -C to confirm. • printf '%s\0%s' foo bar | hexdump -C writes foo.bar (where . is a NUL byte) to stdout. Note that \0 inside single or double quotes doesn't expand on its own; only $'\0' quoting would expand it before printf, which would make it act as a terminator.
• printf '%b' 'foo\0bar' does the same thing, but more complicated.
• While it's not possible to include 0 bytes in the list of arguments and env vars passed to the execve() system call, there's nothing preventing builtin commands or shell functions from supporting them in their arguments. zsh does, and so does it support passing NUL bytes on its command lines. It also has a mkdir builtin, but obviously passing NUL bytes won't help since the mkdir() system call also takes a NUL delimited string as argument (actually zmodload zsh/files; mkdir $'\0' currently exhibits a bug there). – Stéphane Chazelas Aug 18 '15 at 8:33 • Note that it's possible to pass a NUL byte in a system call, just not for those syscall arguments that are pointers to NUL terminated strings. Example: write(fd, buf, 12) (here the size argument (12) indicates the extent of the (binary as opposed to text) string). Which is why printf '\0\0' does manage to print those 2 NUL bytes (it does a write(1, (char[]) {'\0','\0'}, 2)). – Stéphane Chazelas Aug 18 '15 at 8:46 • @StéphaneChazelas: thanks for the edit of shell special characters. I clarified the bit about sys calls, since your 2nd comment is probably helpful to anyone seeing this concept for the first time. – Peter Cordes Aug 18 '15 at 11:51 • @StéphaneChazelas How would one figure out the length of the string when the kernel itself strips everything after an inbuilt NUL character? I don't think zsh itself supports it because there is no safe way to support it (except weird userspace trickery... except as I said again the kernel itself won't pass the full string) – Paul Stelian Mar 29 '18 at 16:59 • @PaulStelian: That's exactly the point. ZSH can handle NUL internally, but the only thing you can do with such data is write it to file descriptors (with echo or printf piping into something, or a here document), not put it on command lines. The Unix system calls for writing/reading data to/from file descriptors use buffer + length: write(2) and read(2). And pwrite / pread. Because those system calls have to work with binary data, not strings. – Peter Cordes Mar 30 '18 at 0:21 As you know $var will lead to interpreting the variable. The reason for why the different options work vary:

• the escape ( \$var ): do NOT interpret the very next character as shell functional character. BUT in some cases: give a special meaning (e.g. \n for newline in some contexts) • the single quotes ( '$var' ): everything in single quotes will strictly be nothing but the very string they contain
• the separation of $ ( "$"var ): a single $ will not be interpreted, by placing it in double quotes it is separated from the var part and no interpretation happens • the double quotes ( "$var" ): will actually allow to interpret the variable var: mkdir "\$var" DOES NOT WORK AND IS NOT THE SAME AS THE OTHERS! Please check again! However anything contained inside the quotes will be treated as a single sting. Especially useful when you have special characters in file names, e.g. creating a file with a space in its name: touch "a b" -> single file a b created/updated, touch a b -> two files a and b created/updated.

Other special operators are: redirections and 'heres' > >> < << <<<, process operators & |, boolean operators || &&, and command separators ; and grouping with parentheses ( ), sometimes - but then separated or as first character - the - for stdin or options of commands. Also there is the test command [ and the quotes we already used ' ", as well as recalling previous commands with an exclamation mark ! or comments with a hash #, and the wildcards asterisk * and question mark ? for multiple and single characters. Also note that the current and parent dir are . and .., while the home is set at ~/. I.e. the characters ; & | > < - [ \ ' " ( ) # * ! ? . ~ ^ { }, , newline, space, tab (and the other blank characters in single-byte locales) should be looked at twice but not all of them are "dangerous" on the same level. I hope I forgot none as there are quite a bunch of them.

• Note that from your list, . and - are not special in that quoting them won't make any difference (except maybe in things like [[ "-e" = x ]]). Quotes can also be used to escape shell language keywords (like "for" or \for to run the for command as opposed to start a for loop) or aliases (\rm to make sure to run the rm command in case there's a rm alias defined). – Stéphane Chazelas Aug 18 '15 at 8:25

In filenames, '/' is prohibited because it is a directory separator. That is the only reason. And if you hand-edit a filesystem, you might even be able to create a file with '/' in the name (not recommended, as you won't be able to do much with it).

The NUL character cannot be used as part of the filename, because the relevant system calls use C language string-passing conventions, and NUL is the terminator for such a string. So it cannot be interpreted as part of the name.

Note that creating a file called \0 is not the same as creating one containing a NUL - the former is a filename containing the two characters '\' and '0'.