6

When I run

for i in {1..2000}; do sleep 0.1 && echo $(netstat -s | grep -i syns); done | uniq

i can see in realtime when expression $(netstat -s | grep -i syns) changes.

But when I want to modify output of pipe:

for i in {1..2000}; do sleep 0.1 && echo $(netstat -s | grep -i syns); done | uniq | while read line; do echo $(date) $line; done

the realtime behaviour is lost. How can I use output of uniq for further manipulations, in particular, append date to each input line change?

Improve this question
4
  • Of course uniq doesn't produce what you call "realtime" output. It emits only unique lines - to do that it has to process the entire set of input lines and remove duplicates. It can't do that until the input is complete, so you won't see any output until the input pipe is closed.
    – Andrew Henle
    Jul 14, 2016 at 9:17
  • Form man uniq: Note: uniq does not detect repeated lines unless they are adjacent. You may want to sort the input first...
    – choroba
    Jul 14, 2016 at 9:19
  • both of these comments are great but missing the point... it should be semi-realtime because it prints the uniq lines that are queued up once it finds a difference, because as both of them have said, it only looks when they're adjacent. But using sort will do the opposite... it will make it impossible to be realtime until after the sort step because sort must process everything before printing anything.
    – Peter
    Jul 14, 2016 at 9:24
  • @AndrewHenle when uniq is last in piped expression, I see realtime behavior. I don't do sort or count exactly for this purpose
    – danbst
    Jul 14, 2016 at 9:28

1 Answer 1

Reset to default
7

Perhaps you only have to change the buffering:

# buffering totally off
stdbuf -i0 -o0 -e0 command

# buffering per line only
stdbuf -oL -eL command

It works for me here... your results may vary. If it doesn't work, try putting it before different commands (not while or read though, since they are bash builtins).

for i in {1..2000}; do
    sleep 0.1 && echo $(netstat -s | grep -i syns)
done \
| stdbuf -oL -eL uniq | while read line; do
    echo $(date) $line
done
Improve this answer
1
  • wow, stdbuf is exactly what I expected to exist!
    – danbst
    Jul 14, 2016 at 9:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .