1

To describe the issue, I created a simple C program.

root@u2004:~# cat test.c 
#include <stdio.h>

int main(){
    printf("output 1\n");
    fprintf(stderr, "error 1\n");
    printf("output 2\n");
    fprintf(stderr, "error 2\n");
    return 0;
}
root@u2004:~# gcc test.c 
root@u2004:~#

When I run it, I got this output:

root@u2004:~# ./a.out 
output 1
error 1
output 2
error 2
root@u2004:~#

As you can see, the output order in the console is the same as the statements defined in the program. Now I want to redirect both the stderr and the stdout to a single file, and keep the output order. Below is what I've tried:

root@u2004:~# ./a.out > out 2>&1
root@u2004:~# cat out
error 1
error 2
output 1
output 2
root@u2004:~# 
root@u2004:~# cat <(./a.out)
error 1
error 2
output 1
output 2
root@u2004:~#

As you can see, in both cases, the error messages goes first, and then the normal output. This is not the same order as defined in the program. How to keep the output order when redirecting both stdout and stderr to the same file?

1 Answer 1

3

That's because of buffering in the program (in the C library). The default behaviour is that output to stdout is line buffered if it goes to a terminal, but fully buffered if it goes to a file. And stderr is always unbuffered by default.

So, since you're printing full lines, line buffering isn't evident, but when redirected to a file, the output that goes to stdout is buffered until a whole block is collected (a few kB), and only actually written then. (You could also try to see what happens if you print partial lines.)

You change the buffering behaviour inside the C program with setbuf()/setvbuf(), and from the outside with the stdbuf tool (and a number of others, look up pipe buffering on the site).

E.g. adding setbuf(stdout, NULL) at the start of your program would make stdout unbuffered, and running it with stdbuf -o0 ./a.out would also do that. But that will slow the program down.

Note that in your last example, cat <(./a.out), it's only the stdout of the program that goes through cat, its stderr goes directly to the terminal. If the two go to different places, through another process, it will cause reordering. You'd need cat <(./a.out 2>&1), or ./a.out 2>&1 | cat to have them both go through cat.

6
  • Thanks for the answer. So the pipe operator | only pipe the stdout of its left command, not stderr, right? I don't know that before. Thanks for pointing it out! May 9 at 7:54
  • 1
    @FajelaTajkiya, yep. It's hard to see with plain cat, but e.g. ./a.out | cat -n would make it evident. You'd only get line numbers on the stdout lines.
    – ilkkachu
    May 9 at 8:00
  • 1
    the idea there is that you can do stuff like tar -c ... | gzip > mydata.tar.gz, and any errors from tar go to the terminal, instead of being compressed and stored with the data (which would probably make the archive broken and useless)
    – ilkkachu
    May 9 at 8:01
  • Thanks for the clarification ! May 9 at 8:12
  • @FajelaTajkiya, so, I also wondered a bit what it it is you're printing. It might be relatively common for someone to want to redirect stdout to /dev/null, to only get the errors, so it would be nice for the error output to be useful even without the stdout output. Maybe you do have it that way, and the problem with reordering is just the aesthetics. Just thinking out loud.
    – ilkkachu
    May 9 at 8:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.