12

Consider: (Using Linux/BASH, unsure about UNIX Proper)

I am expecting a 2 error when arguing a file that does not exist...

grep "i am here" real-file

# Returns: 0 (via: echo $?)

grep "i am not here" real-file

# Returns: 1

grep "i am not here" not-a-file

# Returns: 2 (No such file or directory)

ls real-files

# Returns: 0

ls not-files

# Returns: 2 (No such file or directory)

...Those make sense, but...

cat real-files

# Returns: 0

cat not-files

# Returns: 1 (No such file or directory)

...Shouldn't "No such file or directory" be STDERR with exit-status 2?

Status 2 came with grep and ls non-files, but cat returns a 1 with the identical error message.

I recognize that grep could have three results (each above), but I think ls would only have two as would cat. So, two possible results couldn't be the reason for cat because it isn't so with ls.

Is this a problem in the BASH code? Do we need to call Linus and Richard? If this is correct, please help me understand why.


After accepting an answer, I'd love an answer that expands on the original Question since this is Linux/BASH, not UNIX Proper: Does UNIX (ie on a Mac) do the same thing or similar things?

  • 1
    cat will encounter one kind of error (no file) while grep have two kind of errors to deal with (file existance and patern existance) ! even ls should be like cat . – Jonah May 31 at 0:01
  • So, the answer has been edited, I've added the key point in bold. Let me know if anything is unclear – Sergiy Kolodyazhnyy May 31 at 0:19
  • @Jonah I doubt it's the only kind of error it can experience. What about "Permission denied"? Also, ls returns status 2 on "No such file or directory", so it's not like cat. – JoL Jun 1 at 17:25
  • @JoL but what forbith file permession and file existance from being categorized under "unable to open file" ? if already we have "test" tool that we use before any operation on a file ! and in this case if ls is not like cat then cat is like ls cause it'll encounter permission denied too !!! – Jonah Jun 1 at 17:51
  • @Jonah I don't really understand what you're trying to say in the first part of your comment. About cat and ls being alike, I'm talking specifically in the context of the question which is regarding the exit-status of the error "No such file or directory". ls uses status 2, cat uses status 1. They're not alike. You gave the justification that it's because cat has "one kind of error", and grep has 2. All I'm saying is that there's more. – JoL Jun 1 at 18:14
19

Let's address some of the parts bottom to top and get rid of non-important parts first:

Is this a problem in the BASH code?

No, cat is entirely separate binary application and unrelated to bash. In some shell configurations, as pointed out by Stephane Chazelas , cat can be a built in, but even then return status of an application is entirely separate from whether or not that application is related to shell or not.

Do we need to call Linus and Richard? If this is correct, please help me understand why.

No, it is not a problem, and Linus and Richard are completely unrelated here. Well, correction: unless they someday declare that exit() and errno absolutely MUST be related and for some odd reason we must follow all their technical decisions.


It is entirely OK that both applications return different exit status , because POSIX specifications have no explicit restrictions or assignments that say "This non-zero exit status shall mean this and that".

POSIX documentation of exit syscall states:

The value of status may be 0, EXIT_SUCCESS, EXIT_FAILURE, or any other value, though only the least significant 8 bits (that is, status & 0377) shall be available to a waiting parent process.

This means that only status 0 has assigned meaning, which is assigned to EXIT_SUCCESS as specified in stdlib.h specs. But this is POSIX spec, how does Linux spec compare ? Well, it's about the same: Linux exit(3) manual doesn't even specify what possible values may be.

Note also that it says "may be" and not "shall be", in the sense that it is not absolutely required for application to exit with specific value, even in case of errors. Your application can encounter an error or failure and still return 0 upon exit.

However, POSIX spec for each portable application does specify EXIT STATUS section, which is specific to each application. Again, there's no pattern besides 0 for success and non-zero for anything else. For instance, POSIX cat specs requires:

The following exit values shall be returned:

0    All input files were output successfully.

>0   An error occurred.

For grep we have:

The following exit values shall be returned:

 0    One or more lines were selected.
 1    No lines were selected.
>1    An error occurred.

Within Linux context, cat(1) doesn't explicitly state these status values, but GNU documentation does. grep(1) manual mentions using exit code of 2, but even then acknowledges that POSIX implementation only requires greater-than-zero condition for errors and urges "...for the sake of portability, to use logic that tests for this general condition instead of strict equality with 2."


It is worth mentioning that in some cases there is assumption that exit() status value equals to errno value. I couldn't find any documentation or reference that would suggest POSIX requires that so far. In fact, it's the opposite. Note, that POSIX exit spec and Linux exit(3) man page do not explicitly state that exit status has to somehow match errno. So the fact that return value of 2 in GNU grep matches ENOENT error value 2 is purely coincidental.

In fact, if we consider errno.h specific integer value isn't even required to be assigned and is implementation dependent. So there could very well be Unix-like implementation that treats ENOENT as integer 2. But again - that's entirely unrelated, because exit status and errno are separate things.

In conclusion:

The fact that cat returns different exit code than grep is appropriate and consistent with the spec for those applications. Exit code meaning is not fixed, and is dependent on each individual application (unless it's a POSIX application like cat or grep, in which case for the sake of portability they should follow).

To quote GNU OS documentation: "The most common convention is simply 0 for success and 1 for failure. Programs that perform comparison use a different convention: they use status 1 to indicate a mismatch, and status 2 to indicate an inability to compare. Your program should follow an existing convention if an existing convention makes sense for it."

| improve this answer | |
  • 5
    ksh93 has a builtin cat (enabled if /opt/ast/bin is ahead of /bin in $PATH or after builtin cat). cat is also builtin (sort of) in busybox sh. bash also ships with an loadable cat builtin. Some distributions make it available in a bash-builtins package. – Stéphane Chazelas May 31 at 11:10
  • 2
    "it is typical for exit status to correspond with errno": citation needed. – John Bollinger May 31 at 17:54
  • 3
    One should always consider the "namespace" or "value space" of errno and an exit status to be completely and entirely and totally separate. They are different things, their values have different meanings, and they are entirely unrelated. – Greg A. Woods May 31 at 18:50
  • 2
    “it is typical for exit status to correspond with errno” — No, on the contrary, it is uncommon, and pretty much guaranteed to be a mistake. (And you can tell Larry I said so.) “some GNU tools in Linux context return exit status 2 for no file” — No, ls returns 2 because that's the error code that it returns for any error, and the fact that ENOENT has the value 2 and is a common error for ls to encounter is just a coincidence. – Gilles 'SO- stop being evil' May 31 at 19:43
  • 3
    Worth pointing out that errno codes could be greater than 255. Any of them that were, couldn't be passed through exit() and waitpid() without data loss. (On the computer where I'm typing this, all of the errno codes are safely lower than 255, but the only constraints ISO C and POSIX place on the values of errno codes are that they must be positive and fit in an int; see pubs.opengroup.org/onlinepubs/9699919799/basedefs/errno.h.html .) – zwol Jun 1 at 15:42
18

The GNU coreutils documentation of cat:

An exit status of zero indicates success, and a nonzero value indicates failure.

...a non-zero exit status indicates a failure, nothing more and nothing less.

The man page of grep:

Normally the exit status is 0 if a line is selected, 1 if no lines were selected, and 2 if an error occurred. However, if the -q or --quiet or --silent is used and a line is selected, the exit status is 0 even if an error occurred.

And the man page of ls:

Exit status:
0   if OK,
1   if minor problems (e.g., cannot access subdirectory),
2   if serious trouble (e.g., cannot access command-line argument).

Your results are consistent with the documentation.

| improve this answer | |
  • 3
    You should point to and quote the standard (ie POSIX) spec instead of the GNU documentation. The original answer had no GNU, coreutils or linux tag. For instance, the standard does not say that the exit status of grep should be exactly 2 in case of error, but just greater than 1. And the standard does not require the exit status of ls to differentiate between "minor problems" and "serious trouble". No, I will not write another answer -- I will just downvote yours ;-) – Uncle Billy May 31 at 3:20
  • 4
    @UncleBilly The question was not tagged with linux (and not with posix either), but has "Linux" in its text so I assumed GNU utilities were used (which they probably are, maybe not), at least the exit codes match the behaviour of the GNU tools. – Freddy May 31 at 4:06
  • 3
    @UncleBilly, the original question said "Using Linux"... And with that said, the assumption of GNU tools is quite natural, and explains the results they got, so you might want to tune that downvote-trigger a bit. – ilkkachu May 31 at 10:15
  • 2
    @JesseSteele, in general, don't read too much into the exit codes other than == 0 and != 0, unless the documentation of the tool in question says there's a difference. As you saw, they're not as consistent as one might hope. (Though >= 127 is commonly generated by the shell for errors and programs terminated by a signal.) – ilkkachu May 31 at 10:32
  • 2
    @UncleBilly the Linux tag wouldn't be appropriate here. That tab is for questions asking about the kernel and interlans of a Linux system and not to just indicate that a question is about Linux. Please take the time to learn local tagging customs before presuming to instruct others. – terdon May 31 at 10:57
9

The exit status of a program must follow a few rules, and beyond these rules there are common conventions. None of these conventions are related to the low-level error that caused the program to exit. It's possible to write a program that exits with a certain error code it decided to exit because a file didn't exist, and a different error code if it decided to exit because it was denied permission to access a file, and a different error code if a directory component of a path turned out to be a non-directory, and so on, but that would be extremely unusual and difficult to arrange.

The exit status of a program is an integer value. On POSIX systems, the type of this value is int, which usually ranges from -231 to 231+1. However most of this range is unusable in practice for several reasons. First and foremost, for historical reasons, most interfaces that allow a program to observe the exit status of its children only return the lower 8 bits of the exit status, which is a value between 0 and 255. This includes the system functions wait and waitpid¹ as well as the exit status in the shell². So for almost all intents and purposes, an exit status is an 8-bit value.

The value 0 is treated as a success and every other value is treated as a failure. This is the case in the shell, where boolean operators, if and while constructs and anything else that involves a notion of true/false considers the exit status 0 to be true and any other status to be false. This is also the case with make, where a nonzero exit status causes the build to stop with an error message and an error status. You can quibble whether it's a convention (since the author of a program technically can return whatever status it wants, and “success” and “failure” aren't defined formally anyway), but in practice, a program that exits with the status 0 is considered to have succeeded and a program that exits with another status (1–255) is considered to have failed.

A further feature of the shell specifically that limits the range is that the exit status in the shell (observed via $?) encodes other information:

  • 126 indicates that the command name is an existing file which is not executable.
  • 127 indicates that the command name could not be found.
  • 128+N traditionally (and still today in most shells) indicates that the command exited with signal N. A few shells use a different range, always beyond 128.

Therefore, in practice, programs cannot usefully use an exit status beyond 125. This leaves the values 1–125 to express different errors.

There is a somewhat widespread, but far from universal convention that larger values are treated as “worse” failures. In particular, for search commands such as grep, 1 indicates “not found” and 2 or more indicates some error that prevented the search (e.g. file not found, as opposed to file found but does not contain the search string). Similarly, comparison commands such as cmp and diff exit with the status 0 to mean “identical files”, 1 to mean “different files” and 2 or more to mean “the comparison could not be completed due to an error”.

A few programs define different error codes for different errors, for example sendmail and some other mail-related programs (values defined in sysexits.h), rsync, curl, wget.

By far the most common convention for error codes is 0 for success, 1 for failure. The C and C++ programming languages define EXIT_FAILURE as an exit status code to use to report a failure if there's no particular reason to choose a specific value, and EXIT_FAILURE is 1 on most systems.

Errors such as “No such file or directory”, “Permission denied”, “Not a directory” and so on have a numerical encoding under the hood: they're errno values, returned by system functions to indicate what went wrong. Errno values are generally not useful as exit status from programs. They encode the minutiae of what went wrong, rather than what it means for the specific program. For example wget's exit status distinguishes “parse error in options” (usually no underlying system error), “local input/output error” (regardless of the underlying system error), “network failure” (which would largely share the same system errors as local I/O), etc. Knowing whether wget failed due to a network error or to a local file error is more useful than knowing whether it failed due to a broken pipe (writing to a pipe, or closed connection on a network socket?) or to a permission error (unable to read the configuration file, or network access denied by a local policy?).

It's somewhat uncommon for return statuses to follow errno values. It does happen, especially with Perl scripts, due to the way Perl's die function works. But it's a bad idea, not only because as I mentioned above the errno value is rarely the most useful part of the information, but mostly because there's no reason why errno values would be in the range 1–125. Fortunately I don't know of any system where errno values are outside the range 1–255, so at least exit(errno) (or Perl's die) won't exit with a value which is a multiple of 256, which as we saw above would convey success. But on Linux, for example, they do reach 126, and a program that exited with exit(errno) with errno == ERFKILL (“Operation not possible due to RF-kill”) would be undistinguishable from the shell from a program that died of SIGILL (illegal instruction).

¹ waitid grants access to the full int value via infop->si_status.
² Via $? or otherwise. For example, if exit256 is a program that exits with exit(256), the shell command if exit256; then echo "exited with 0"; fi prints “exited with 0”.

| improve this answer | |
  • 1
    test/[ too in the 0=true, 1=false, 2=error team. The OpenSSH client uses an exit status of 255 when it itself faces an error, which is kinda useful since otherwise it returns whatever the remote command returned, which could be any of the "regular ones" incl. the shell's codes for signal-induced termination. – ilkkachu May 31 at 20:13
  • 2
    128+N is not a POSIX requirement, some shells do 256+N or 384+N. Some non-POSIX shells encode the signal some other way. See also Default exit code when process is terminated? and Is it possible to determine the signal received by last running application? (and also relevant here: What is the min and max values of exit codes in Linux?) – Stéphane Chazelas Jun 1 at 18:30
  • Amazing treatise. That explains why I saw EXIT_FAILURE in the code when I looked on it from the GNU site. It was probably inheriting the 1 value from the C language system. This is long, but very educational for what's under the hood in the GNU/Linux shell. – Jesse Steele Jun 3 at 0:12
  • Apparently, in bash (exit 256) is such a program as exit256? – jrw32982 supports Monica Jun 3 at 18:00
  • @jrw32982supportsMonica No, not in bash: bash takes the lower 8 bits of the error code, so exit 256 exits with the status 0. In dash and zsh, (exit 256) does call exit(256), but you still get 0 in $? in the parent shell process. – Gilles 'SO- stop being evil' Jun 3 at 18:48
3

Success

In general and as stated in the POSIX spec only an exit status of 0 is defined as:

A value of zero (or EXIT_SUCCESS, which is required to be zero) for the argument status conventionally indicates successful termination. This corresponds to the specification for exit() in the ISO C standard. The convention is followed by utilities such as make and various shells, which interpret a zero status from a child process as success. For this reason, applications should not call exit(0) or _exit(0) when they terminate unsuccessfully; for example, in signal-catching functions.

Being specified in the POSIX spec means it must apply to all UNIX, and, most probably Linux also will follow that specification.

Failure

Any other value of the exit status (by contrast) is a failure.

That is all that can be said and all the meaning the exit status could have.

There is no connection between the exit status value and any other list of values, not even the errno usually set by C functions. That ENOENT happens to be 2 has no relation to an exit status of 2, nor could be expected to be so.

Why would an utility like [ (which require no file to work) to report an error on a file?

In fact, the POSIX spec for [ only defines this:

The following exit values shall be returned:

0        expression evaluated to true.

1        expression evaluated to false or expression was missing.

>1       An error occurred.

And, as an example from man bash:

All builtins return an exit status of 2 to indicate incorrect usage,
generally invalid options or missing arguments.

Which has no relation to reading a file.

Shell

Is this a problem in the BASH code?

No, not at all, as long as SUCCESS is signaled by 0.

Do we need to call Linus and Richard?

Why bother them?

Why?

If this is correct, please help me understand why.

The only reasoning I could provide (not having asked the developers that decided that was a "reasonable" requirement) goes with the old saying:

Be conservative in what you send, be liberal in what you accept

leaving the maximum flexibility to each application developer on how (and which) errors should be reported.

Does UNIX (ie on a Mac) do the same thing or similar things?

Yes, if a Mac wants to be certified as UNIX, it should follow the POSIX spec.


Edit:

If what you need is a number, In freeBSD:

grep "i am here"     real--file; echo "$?"     # ==> 0
grep "i am not here" real--file; echo "$?"     # ==> 1
grep "i am here"     not-a-file; echo "$?"     # ==> 2
cat  real--file                                # ==> 0
cat  not-a-file                                # ==> 1
| improve this answer | |
  • Very useful, except the UNIX part at the end: the "same thing" would be exit 1. It would need an exit status. You didn't answer my Question there, you answered your Answer. Give an exit number and I'll upvote. :-) – Jesse Steele Jun 2 at 17:49
  • Look @JesseSteele There is no "number" to give you for the UNIX part, nor for BSD, Linux or any OS, that was the whole point of this answer, any number is equally valid. – Isaac Jun 2 at 20:17
  • Are you claiming that Linux and UNIX do not return a number with echo $? after a cat command? I'm asking what the number would be in that part of the question on UNIX. I did get a number for GNU/Linux and it seems you're telling me I did not. Please clarify. – Jesse Steele Jun 2 at 23:45
  • 1
    Isaac, the question centers around the difference in return codes between various commands. Your answer currently points to the specs for return codes in general, which is helpful, but doesn't address what Jesse appears to be confused by here -- the different return codes by different utilities in apparently the same situation. – Jeff Schaller Jun 3 at 0:22
  • 1
    @JesseSteele Are the numbers on the edit what you want? No, I have no Mac's now. Even if that is what you asked for, they are inconclusive, any present or future cat or grep might give some other exit code and still be "perfectly correct". – Isaac Jun 3 at 1:03
1

Depending on your system, cat may be a shell built-in or a separate binary. To see which it is you can run

$ command -V cat

Also, the GNU behavior of cat is indeed correct, to quote POSIX cat(1):

EXIT STATUS

The following exit values shall be returned:

0
All input files were output successfully.

>0
An error occurred.

A relation between errno and exit status is merely coincidental, as errno doesn't need to be in range of the lower 8 bits of the exit code (which are those that POSIX demands to be passed on).

The POSIX '01 compliant SunOS 5.10 here does return 2 though (and no standard between XPG3 and POSIX '01 apparently changed the behavior of this tool):

$ PATH=`getconf -v POSIX.1-2001 PATH`
$ export PATH
$ command -v cat
/usr/bin/cat
$ cat nosuchfile
cat: cannot open nosuchfile
$ echo $?
2

Which indeed happens to be ENOENT on Solaris as well:

$ grep ENOENT /usr/include/sys/errno.h
#define ENOENT  2       /* No such file or directory            */

The manpage does only document >2 however.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.