Is there a file that will always not exist?

Question

Along the lines of /dev/null (path to an empty source/sink file), is there a path that will never point to a valid file on at least Linux? This is mostly for testing purposes of some scripts I'm writing, and I don't want to just delete or move a file that doesn't belong to the script if it exists.

Answer 1

As an alternative, I would suggest that your script create a temporary directory, and then look for a file name in there. That way, you are 100% certain that the file doesn't exist, and you have full control and can easily clean up after yourself. Something like:

dir=$(mktemp -d)
if [ -e "$dir"/somefile ]; then
    echo "Something is seriously wrong here, '$dir/somefile' exists!"
fi
rmdir "$dir"

You can write the equivalent code in any language, the vast majority (all?) higher level languages will have some dedicated tool to handle creating and deleting temporary directories. This seems like a far safer and cleaner approach than trying to guess a file name that should not exist.

Answer 2

/dev/null/foo cannot exist, unless /dev/null is a directory.

POSIX requires /dev/null to be "an empty data source and infinite data sink". I'm not sure if it's totally impossible to have a directory with these characteristics. Nevertheless I think it's pretty safe to assume /dev/null is not a directory in your *nix.

Note if you try to open /dev/null/foo then you will get ENOTDIR (not a directory), not ENOENT (no such file or directory). This may or may not be acceptable for your testing purposes.

Answer 3

Taking a page from any cryptography handbook, if you generate a large enough random string, it will only exist on any given system with such an insignificant probability that the possibility of a hit can be ignored.

E.g. assuming you have a working /dev/urandom (and you should), something like this would generate in f a valid filename based on a 128-bit random number:

f=/$(head -c 16 /dev/urandom |base64 |tr / ,)

The output is something like /B90sYd,aNrcw7d7Itcb8fQ==. (The leading slash is fixed and on purpose to make it an absolute path. The tailing == are also fixed and due to Base64 padding. They can be ignored.)

A system generating random file names at the rate of 1 trillion / second would take trillions of years to generate the one generated by your script. Note that any collision isn't enough, so the birthday attack doesn't apply. This is basically the same as brute forcing a 128-bit symmetric key.

See also e.g.: How long would it take to brute force an AES-128 key?

Note that this requires a working /dev/urandom. It won't work if someone has replaced that with a static file containing some known string, like e.g. the three bytes \x86\x89\x9e, which when Base64-encoded, produce the string home; or if you're in e.g. a chrooted context where /dev/urandom isn't available. Also, e.g. embedded systems with no real means to initialize the system's RNG may face issues. Don't use this in situations like that, but also don't generate any cryptographic keys in situations like that.

---

As an alternative, you could possibly use the empty string. At least on Linux, trying to use it as a filename just gives an error:

$ cat ""
cat: '': No such file or directory
$ touch ""
touch: cannot touch '': No such file or directory

(As an aside, I find the error it gives (ENOENT) somewhat amusing. One might think it'd say the name is invalid, instead of that it doesn't exist.)

Note that if you put that in a variable, you really need to remember the quotes when expanding it! E.g. f=; cat $f would just read from stdin.

$ f=
$ cat "$f"
cat: '': No such file or directory
$ touch "$f"
touch: cannot touch '': No such file or directory

However, if you do cd "" in the shell, it just changes to the current directory. POSIX says that "If [the given path] is an empty string, the results are unspecified.". All shells I tried explicitly use the current path in the chdir() call, e.g.:

/tmp$ strace -etrace=chdir zsh -c 'cd ""'
chdir("/tmp")                           = 0
+++ exited with 0 +++
/tmp$

I got this idea based on an earlier (now deleted) answer. This may or may not be system-specific, I only tried on Linux. Caveat emptor.

---

Also, as mentioned in comments, if you don't care which exact error you get, or use something that doesn't even tell you, e.g. [ -f ... ] or [ -e ... ] in the shell, you could just create an over-long filename.

On pretty much all filesystems, the maximum length of a single file is 255 or less (see the table in Comparison of file systems on Wikipedia). A full path can be longer, but a single file name of 256 bytes is impossible, and gives ENAMETOOLONG:

$ f=$(printf %256s x | tr ' ' x)
$ touch "$f"
touch: cannot touch 'xxx...xxx': File name too long

But doing if [ -e "$f" ]; then ... works without error (and the test fails) in all shells I tried.

(The POSIX definition says that -e is "False if pathname cannot be resolved", but doesn't explicitly mention diagnostics. So perhaps some implementation could give an error in some situation, I'm not sure. Do tell if you find such a case.)

_{(The table in Wikipedia does mention two Linux filesystems with a higher per-file length, but I doubt they're used much nowadays, and I also understand Linux has a 255-byte limit in general, regardless of the filesystem.)}

Answer 4

Since process IDs are never negative, /proc/-1 will never exist.

This also works even if your variant of Unix doesn't support the procfs (or if it's not mounted)!

---

A similar method using file descriptors: /dev/fd/-1

I suppose some variant of Unix might theoretically allow a file descriptor of -1, but that is going to break a lot of existing code.

---

There are probably many other such possibilities with auto-generated filesystems.

Answer 5

terdon has already given a good answer: use mktemp -d. This post looks at the question in a more fundamental way, and explains why this command is usually the best answer.

There is no constant path that will always not exist. But based on your sentence about testing scripts, it sounds like a variable path is just as good, provided the script can generate it itself.

The only way to generate such a path is to keep generating paths, until the generated path does not exist. Unfortunately, this can lead to a race condition: some other program might be create the file after you check whether it exists, but before you perform whichever operation depends on the file not existing.

The best way to avoid this race condition is to use a path that other programs are supposed to avoid, like a path under /tmp. Unfortunately, /tmp tends to be world-writable, so now you have an even bigger problem: some other user might create the file.

What you really want is a temporary directory that other programs are supposed to avoid and other users do not have access to. Even better if the directory is empty, so you can just use any path under that directory.

mktemp -d creates a directory meeting all the criteria in the previous paragraph, while defending itself against its own race conditions. When you have finished, you can use rmdir to remove the directory:

dir="$(mktemp -d)"
# Use "$dir"/foo as a nonexistent path.
rmdir "$dir"

Answer 6

I like to keep things simple. Instead of using a path that cannot exist, I use one that I would never create, nor can I imagine anyone creating.

My tests that need a non-existent path use something like /thisfiledoesnotexist. I consider this reasonable, because:

• The filename is self-documenting.
• It takes root permissions to create this file, so it's unlikely to get created by accident.
• I would never create it.
• I can't imaging why anyone else ever would. Linux sysadmins like to keep the root directory clean

If you are truly paranoid, then your test harness can first test that the file does not exist, failing if it does. Only if the file does not exist would it proceed with the test.

Answer 7

The empty path, "", cannot exist in Linux or POSIXy systems. In Linux, empty path always fails with ENOENT (see man 7 path_resolution), on other systems the error might be different; POSIX only says that it must not be resolved successfully.

Empty pathname

In the original UNIX, the empty pathname referred to the current directory. Nowadays POSIX decrees that an empty pathname must not be resolved successfully. Linux returns ENOENT in this case.

^(source)

A null pathname shall not be successfully resolved.

^(source)

Answer 8

Let's assume the criteria are:

• test should work without write access to the device (so no temporary folder);
• must return E_NOENT (so no using a known-existent filename as a parent folder, as that gives E_NOTDIR).

For this case, the deprecated C library functions char *tmpnam(char *str) and char *tempnam(const char *dir, const char *pfx); both generate and return a short, valid temporary filename which is guaranteed not to exist at that point in time.

It's possible that this may be sufficient for your use case, but of course, the methods are deprecated for a reason. There is a race condition between running tmpnam and checking for the file's existence, during which the file could be created. A malicious attacker can also deliberately create filenames with far higher chance of collision than 1 in TMP_MAX. Even with multiple non-malicious processes, the birthday problem arises.

Plus, these library calls require C, which your tests are unlikely to be written in. We could implement similar functionality in the test code: generate a filename, see if it exists, repeat until one doesn't exist.

However, a naive implementation would not only be vulnerable to the race condition above, but also have a very lengthy fail state: if E_NOENT is never correctly returned, so every file appears to exist, it'll keep trying indefinitely. That's exactly the wrong kind of fail state, when writing a test for E_NOENT: it means detecting failure takes forever!

However, as @ikkachu writes, with suitable creation of longer filenames (eg a UUID, or the approach ikkachu suggests), it should be cryptographically infeasible for our random file to already exist, so we only need to try once. It's tempting to try twice, just in case. But that's only equivalent to adding a single extra bit of entropy to our filename's random length.

Answer 9

If you strictly enforce the file requirement, you can read the absolute path of the current working directory and then use it to check for a regular file.

To work around the possibility of the script deleting the current working directory and creating a file with the same path/name before doing the check, perhaps we can consider the parent directory of the script itself.

A trivial case may be to check for the path "/" which is always guaranteed to be a directory, never a regular file.