5

When run from a terminal, hexdump does not react to a single ^D at the beginning of the line, as cat, od, bc, etc. do, unless there was no input yet:

prompt% hexdump -C
<control-D>
prompt% hexdump -C
hello
<control-D><control-D> # a single ^D won't do
00000000  68 65 6c 6c 6f 0a                                 |hello.|
00000006

I can reproduce this on both fedora 28 and debian 9. Despite hexdump being provided by bsdmainutils on Debian, this does not happen on bsd.

Is this just a bug? Are there any other programs known to exhibit this behavior?

3

Thanks to @JdeBP for the hint, I was able to create a small testcase that does the same as hexdump:

#include <stdio.h>

int main(void){
        char buf[64]; size_t r;
        for(;;){
                printf("eof=%d, error=%d\n", feof(stdin), ferror(stdin));
                r = fread(buf, 1, sizeof buf, stdin);
                printf("read %zd bytes, eof=%d, error=%d\n",
                        r, feof(stdin), ferror(stdin));
                if(!r) return 0;
        }
}

When run on a glibc based system (typical linux desktop).

prompt$ ./fread-test
eof=0, error=0
<control-D>
read 0 bytes, eof=1, error=0

prompt$ ./fread-test
eof=0, error=0
hello
<control-D>
read 6 bytes, eof=1, error=0
eof=1, error=0
<control-D>
read 0 bytes, eof=1, error=0

When run on bsd, solaris, busybox (uclibc), android, etc:

prompt$ ./fread-test
eof=0, error=0
hello
<control-D>
read 6 bytes, eof=1, error=0
eof=1, error=0
read 0 bytes, eof=1, error=0

Based on my inexpert interpretation of the standard, this looks like a bug in glibc (the GNU C library).

About fread:

For each object, size calls shall be made to the fgetc() function and the results stored, in the order read, in an array of unsigned char exactly overlaying the object.

About fgetc:

If the end-of-file indicator for the input stream pointed to by stream is not set and a next byte is present, the fgetc() function shall obtain the next byte

It seems that glibc will try to "obtain the next byte" even if the eof indicator is set.

Indeed, it actually is a bug in the GNU C library, not present in the BSD or musl C libraries. It was known about in 2005. Ulrich Drepper closed the bug report without fixing the bug in 2007. It was discussed in 2012, where it was noted that other C libraries did not and do not have this behaviour, that the 1999 C Standard is quite specific about it, and that Solaris even has a special mechanism for this which is invoked when c99 is used as the compiler instead of cc.

It was finally fixed in 2018. The fix is in version 2.28 of the GNU C library. The current "stable" version of Debian, version 9, is on version 2.24 of the GNU C library, and this bug thus continues to manifest itself, 14 years after being reported.

As noted in the GNU C library discussions, there is the possibility of softwares that were written to require the quirks of the GNU C library without regard to other C libraries such as musl or the behaviour on other platforms. However, in the aforementioned discussions over the years no such program was identified. Whereas several programs that are broken by the old GNU C library, to require users to signal EOF twice in succession, have been identified; including amongst others hexdump here and patch on StackOverflow back in 2018.

0

Output of strace

…
read(0, "hello\n", 1024)                = 6
read(0, "", 1024)                       = 0
read(0, "hello\n", 1024)                = 6
read(0, "", 1024)                       = 0
read(0, "", 1024)                       = 0
fstat(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 11), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8dd4cef000
write(1, "0000000   h   e   l   l   o  \\n "..., 72) = 72
write(1, "000000c\n", 8)                = 8
exit_group(0)                           = ?
+++ exited with 0 +++

Here I did hello «ctrl-d» hello «ctrl-d» «ctrl-d» (the second hello is truncated in the log output). So it looks like it is programmed that way.

To run strace

  • Open two terminals
  • In first type ll /proc/self/fd/1
  • In second type strace hexdump -C 2>/dev/pts/«number-at-end-of-output-of-previous-command»
  • After downloading and compiling the code used by e.g. Debian (and removing the references in the code to the "capsicum" routines) on OpenBSD, it exhibits the same behaviour as the native OpenBSD hexdump. I don't know what "capsicum" is, but this could possibly be what makes it behave like this. – Kusalananda May 4 at 9:17
  • Capsicum won't be the problem, because if you compiled the Debian code properly as it is compiled on Debian Linux it patches out the capsicum code with some Debian patches. I suggest that people try building against, say, musl libc on Debian. – JdeBP May 4 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.