3

I'm try to pipe chat log files to some speech output, and want to remove timestamps with cut and remove special characters with tr. Unfortunately, cut seems to halt when using it with tail -f in combination with tr:

//works
$ tail /path/to/chatlogs | cut -b18- 
test
test

//works, too
$ tail /path/to/chatlogs | tr -C -d  '[:alnum:][:cntrl:] '
test
test

// does not work
$ tail -f /path/to/chatlogs | tr -C -d  '[:alnum:][:cntrl:] ' | cut -b18-
//no output

Why? How can I work around this?


It even hangs when piping two times into cut:

$ tail -f file | cut -b5- | cut -b5-
//no output
  • hmmm... your works example seems to be identical to your does not work example. if there is a problem with tail -f it might have to with its checking for new input only every 60 seconds or so. if it were to try to open() the file all the time it would eat a lot of resources. try while <file; do :; done here <file is some regular, readable file, in the terminal and have a look at top in another terminal while it runs. or : >file; until read v; do : ; done <file. – mikeserv Dec 16 '15 at 11:47
  • 3
    The second tail command waits for additional input because of the follow (-f) switch and never exits. – Lambert Dec 16 '15 at 11:52
  • What Lambert said; why are you using -f if you don't want it to wait for more output? – wurtel Dec 16 '15 at 11:55
9

The syntax you use only processes new input on /path/to/chatlogs. Try to run the command and log a new entry in /path/to/chatlogs and see what the output is or try:

tail -1000f /path/to/chatlogs | tr -C -d  '[:alnum:][:cntrl:] ' | cut -b18-

to have the last 1000 lines also processed.

The 'hanging' part is actually the tail process which is waiting for further input to be passed through the pipe.

To disable buffering from the tr command use:

tail -f /path/to/chatlogs | stdbuf -i0 -oL tr -C -d  '[:alnum:][:cntrl:] ' | cut -b18-

The above is used from Turn off buffering in pipe

0

When using multiple pipes, I'm running into stdio buffering issues.

The problem is described in detail at http://www.pixelbeat.org/programming/stdio_buffering/ and there does not seem to be a solution.

  • See my updated answer about buffering. – Lambert Dec 16 '15 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.