Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have become so used to do this: someprogram >output.file

I do it whenever I want to save the output that a program generates to a file. I am also aware of the two variants of this IO redirection:

  • someprogram 2>output.of.stderr.file (for stderr)
  • someprogram &>output.stderr.and.stdout.file (for both stdout+stderr combined)

Today I have run across a situation I have not thought possible. I use the following command xinput test 10 and as expected I have the following output:

user@hostname:~$ xinput test 10
key press   30 
key release 30 
key press   40 
key release 40 
key press   32 
key release 32 
key press   65 
key release 65 
key press   61 
key release 61 
key press   31 
^C
user@hostname:~$ 

I expected that this output could as usual be saved to a file like using xinput test 10 > output.file. But when contrairy to my expectation the file output.file remains empty. This is also true for xinput test 10 &> output.file just to make sure I do not miss something on stdout or stderr.

I am really confused and hence ask here if the xinput program might have a way to avoid its output to be redirected?

update

I have looked at the source. It seems the output is generated by this code (see snippet below). It appears to me the output would be generated by an ordinary printf

//in file test.c

static void print_events(Display    *dpy)
{
    XEvent        Event;

    while(1) {
    XNextEvent(dpy, &Event);

    // [... some other event types are omnited here ...]

        if ((Event.type == key_press_type) ||
           (Event.type == key_release_type)) {
        int loop;
        XDeviceKeyEvent *key = (XDeviceKeyEvent *) &Event;

        printf("key %s %d ", (Event.type == key_release_type) ? "release" : "press  ", key->keycode);

        for(loop=0; loopaxes_count; loop++) {
        printf("a[%d]=%d ", key->first_axis + loop, key->axis_data[loop]);
        }
        printf("\n");
    } 
    }
}

I modified the source to this (see next snippet below), which allows me to have a copy of the output on stderr. This output I am able to redirect:

 //in file test.c

static void print_events(Display    *dpy)
{
    XEvent        Event;

    while(1) {
    XNextEvent(dpy, &Event);

    // [... some other event types are omnited here ...]

        if ((Event.type == key_press_type) ||
           (Event.type == key_release_type)) {
        int loop;
        XDeviceKeyEvent *key = (XDeviceKeyEvent *) &Event;

        printf("key %s %d ", (Event.type == key_release_type) ? "release" : "press  ", key->keycode);
        fprintf(stderr,"key %s %d ", (Event.type == key_release_type) ? "release" : "press  ", key->keycode);

        for(loop=0; loopaxes_count; loop++) {
        printf("a[%d]=%d ", key->first_axis + loop, key->axis_data[loop]);
        }
        printf("\n");
    } 
    }
}

My idea at present is that maybe by doing the redirect the program looses its ability to monitor the key-press key-release events.

share|improve this question

5 Answers 5

It's just that when stdout is not a terminal, output is buffered.

And when you press Ctrl-C, that buffer is lost as/if it has not been written yet.

You get the same behavior with anything using stdio. Try for instance:

grep . > file

Enter a few non-empty lines and press Ctrl-C, and you'll see the file is empty.

On the other hand, type:

xinput test 10 > file

And type enough on the keyboard for the buffer to get full (at least 4k worth of ouput), and you'll see the size of file grow by chunks of 4k at a time.

With grep, you can type Ctrl-D for grep to exit gracefully after having flushed its buffer. For xinput, I don't think there's such an option.

Note that by default stderr is not buffered which explains why you get a different behaviour with fprintf(stderr)

If, in xinput.c, you add a signal(SIGINT, exit), that is tell xinput to exit gracefully when it receives SIGINT, you'll see the file is no longer empty (assuming it doesn't crash, as calling library functions from signal handlers isn't guaranteed safe: consider what could happen if the signal comes in while printf is writing to the buffer).

If it's available, you could use the stdbuf command to alter the stdio buffering behaviour:

stdbuf -oL xinput test 10 > file

There are many questions on this site that cover disabling stdio type buffering where you'll find even more alternative solutions.

share|improve this answer
2  
WOW :) that did the trick. thank you. So in the end my perception of the problem was wrong. There was nothing in place to inhibt redirection, it was simple Ctrl-C stopped it before the data was flushed. thank you –  humanityANDpeace Dec 28 '12 at 11:55
    
Would there have been a way to prevent the buffering of stdout? –  humanityANDpeace Dec 28 '12 at 11:55
1  
@Stephane Chazelas: thanks a lot for your detailed explanation. In addition to what you've already said i found out that one can set the buffer to unbuffered with setvbuf(stdout, (char *) NULL, _IONBF, NULL). Maybe this is also of interest!? –  user1146332 Dec 28 '12 at 12:18
4  
@user1146332, yes, that would be what stdbuf -o0 does, while stdbug -oL restores line buffering like when the output goes to a terminal. stdbuf does force the application to call setvbuf using a LD_PRELOAD trick. –  Stéphane Chazelas Dec 28 '12 at 12:23
    
another workaroudn: unbuffer test 10 > file (unbuffer is part of the expect tools) –  Olivier Dulac Dec 28 '12 at 15:40

A command can directly write to /dev/tty preventing regular redirection to happen.

$ cat demo
#!/bin/ksh
LC_ALL=C TZ=Z date > /dev/tty
$ ./demo >demo.out 2>demo.err
Fri Dec 28 10:31:57  2012
$ ls -l demo*
-rwxr-xr-x 1 jlliagre jlliagre 41 2012-12-28 11:31 demo
-rw-r--r-- 1 jlliagre jlliagre  0 2012-12-28 11:31 demo.err
-rw-r--r-- 1 jlliagre jlliagre  0 2012-12-28 11:31 demo.out
share|improve this answer
    
Your example makes the point+ answers the question. Yes it is possible. It is of course "unexpected" and uncommen for programs to do so, which at least fooled me in not considering such a thing possible. The answer by user1146332 also seems a convincing way to avoid redirection. To be fair and since both given answers are equally possible ways to avoid redirection of command line program output to a file I cannot select any of the answers I guess :(. I would need to be allowed to select two answers right. Great work, Thank you! –  humanityANDpeace Dec 28 '12 at 11:25
    
FTR, if you want to capture the output written to /dev/tty on a Linux system, use script -c ./demo demo.log (from util-linux). –  ndim Dec 28 '12 at 17:40
    
If you are not running in a tty, but instead in a pty, you can find that by looking at procfs (/proc/$PID/fd/0 etc). To write to the appropriate pty, go to your parent process's fd directory and see if it's a symlink to /dev/pts/[0-9]+. Then you write to that device (or recurse if it isn't a pts). –  dhasenan Dec 28 '12 at 17:48

It looks like xinput rejects output to a file but doesn't reject output to a terminal. To achieve this, probably xinput use the system call

int isatty(int fd)

to check whether the filedescriptor to be opened refers to a terminal or not.

I stumbled upon the same phenomenon a while ago with a program called dpic. After i looked into the source and some debugging i removed the lines related to isatty and everything worked as expected again.

But i agree with you that this experience is very disturbing ;)

share|improve this answer
    
I really thought I had my explation. But (1) looking at the source (the test.c file in the xinput source package) there is no appearance of isatty testing done. The ouput is generated by printf function (I think its the a standard C) one. I added some fprintf(stderr,"output") and this is possible to redirect + proves the whole code is really runned in the case of xinput. Thank you for the suggestion after all it was the first trail here. –  humanityANDpeace Dec 28 '12 at 11:11

In your test.c file you could flush the buffered data using (void)fflush(stdout); directly after your printf statements.

    // in test.c
    printf("key %s %d ", (Event.type == key_release_type) ? "release" : "press  ", key->keycode);
    //fprintf(stderr,"key %s %d ", (Event.type == key_release_type) ? "release" : "press  ", key->keycode);
    //(void)fflush(NULL);
    (void)fflush(stdout);

On the command line you may enable line-buffered output by running xinput test 10 in a pseudo terminal (pty) with the script command.

script -q /dev/null xinput test 10 > file      # FreeBSD, Mac OS X
script -c "xinput test 10" /dev/null > file    # Linux
share|improve this answer

Yes. I even did this in DOS-times when I programmed in pascal. I guess the principle still holds:

  1. Close stdout
  2. Re-Open stdout as console
  3. Write the output to stdout

This did break any pipes.

share|improve this answer
    
“Re-Open stdout”: stdout is defined as file descriptor 1. You can reopen file descriptor 1, but what file would you open? You probably mean open the terminal, in which case it doesn't matter whether the program is writing to fd 1. –  Gilles Dec 28 '12 at 23:08
    
@Gilles the file was "con:" as far as I remember - but yes, I refined point 2 in that direction. –  Nils Dec 29 '12 at 21:54
    
con is the DOS name for what unix calls /dev/tty, i.e. the (controlling) terminal. –  Gilles Dec 29 '12 at 22:15

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.