10

According to this page:

File names in Linux can contain any characters other than (1) a forward slash ( / ), which is reserved for use as the name of the root directory (i.e., the directory that contains all other directories and files) and as a directory separator, and (2) the null character (which is used to terminate segments of text). Spaces are permitted, although they are best avoided because they can be incompatible with legacy software in some cases.

Great, both restrictions make a lot of sense. Since it is clearly possible to forbid the inclusion of certain characters in file names, why were newlines allowed? As far as I can tell their only use is to complicate our scripts. Is there ever a valid reason to have a new line in a file name?

10
  • Why shouldn't it be included? Sure it may complicate your scripts, but so will lots of other characters. – Zoredache Oct 4 '13 at 4:41
  • @Zoredache none of them do so as much as the newline (except perhaps the backslash) and none of them while being so completely pointless. What in the world is the point of allowing these characters given that they cause such complications? – terdon Oct 4 '13 at 4:43
  • 2
    There's no good reason to allow newlines in filenames, but unfortunately we're stuck with them..too late to change now. And the Robustness Principle is subverted because the presence of \n in filenames leads to more fragile scripts as most shell programmers even find dealing with spaces in filenames to be difficult and most of the reset only know about find ... -print0 and xargs -0 (and don't realise, e.g., you can tell bash's built-in read to use NUL as a delimiter with -d $'\0', or that many GNU tools have -0, -z or -Z options for handling NUL-terminated stdin) – cas Oct 4 '13 at 6:02
  • 1
    worse, an extremely common file format (i.e. "one filename/item per line, separated by newlines") is made unreliable by the presence of newlines in filenames. There's not even a reliable way to convert that format to NUL-separated. All you can do is hope/assume that your users are relatively sane and haven't used \n in their filenames. – cas Oct 4 '13 at 6:06
  • 1
    Yes, i understand WHY newlines are legitimate characters in a filename. I just don't think that there's any good reason for them to be allowed, and certainly no good reason to use them. It's possible to login as root to run X and all the usual user GUI apps, but it's a bad idea to do that. same with newlines in filenames - legit but stupid. – cas Oct 4 '13 at 8:33
6

NUL and / has their designated system functions. Other characters does not.

That is the basics of it – the rest is opinions, speculations and history. Heard, read etc. and only included as a filler not a debate or argument:

  • By forbidding certain characters you open up for complexity in the file system itself, which is the same as compromising it.
  • What about which bytes constitute as a newline on various systems? <CR> vs <LF> etc.
  • What if a remote system decides to create a file with newline on a NFS?
  • What if the filename get corrupted whilst the file contents is intact?
  • What if an application encode information in the filename?

And on it goes

  • Is it the systems job to fix bugs in user software?
  • Should a system, on it's root level, protect users from themselves?
  • Should the way the various shells are implemented internally govern a decision as to what file names are considered legal?

The basic operating system doesn't set limitations. Information to and from the system is byte streams. If a byte does not have a special meaning, don't create overhead by adding checks that should be handled in user space.


Anyhow, the biggest issue would most likely be the rather long history where newline, and other control characters, have been allowed.

Another case is what to forbid. You mention newline, but in discussions from the stone-age of UNIX, this has been debated, then also including other characters. Should * be forbidden? What about filenames starting with -? What about DEL and ESC? Should all control characters be forbidden? And so on and so forth.

I can unfortunately not recall any quotes on this topic by the founding fathers or code maintainers.

7
  • 1
    None of your other examples are anywhere near as problematic as \n. They can all be dealt with more easily. The overhead of checking for \n is a valid point but it's not such a big deal. I would say that the overhead is worth avoiding the headaches. I can think of no valid reason why a file name should ever contain \n. – terdon Oct 4 '13 at 14:45
  • "Should the way the various shells are implemented internally govern a decision as to what file names are considered legal?" is the key point. Having trouble dealing with filenames containing newlines is a problem specific to the Bourne shell. C programs have no such difficulty. The Unix kernel was long written before the Bourne shell, so I would guess the filesystem just wasn't originally designed with the shell in mind. – Matt Oct 4 '13 at 20:04
  • @Matt Most modern unices allow any character in file names except NUL and /, but older unices didn't do this. Did the original Unix (the one that predates the Bourne shell) allow exotic characters? – Gilles 'SO- stop being evil' Oct 4 '13 at 22:22
  • Ctrl-O can mess up your tty, and Ctrl-G can make you want to strangle someone. IMO all control characters are problematic. – cas Oct 4 '13 at 22:38
  • @terdon: Yes, I understand your stand, though I'm on the dark side. Anyhow. A very interesting topic that touches the very core of a lot of very interesting topics. "Unfortunately" this is isn't a debate forum so I'll restrain myself from expanding on this any further. – Runium Oct 5 '13 at 17:56

Not the answer you're looking for? Browse other questions tagged or ask your own question.