1

When I run

for i in {1..2000}; do sleep 0.1 && echo $(netstat -s | grep -i syns); done | uniq

i can see in realtime when expression $(netstat -s | grep -i syns) changes.

But when I want to modify output of pipe:

for i in {1..2000}; do sleep 0.1 && echo $(netstat -s | grep -i syns); done | uniq | while read line; do echo $(date) $line; done

the realtime behaviour is lost. How can I use output of uniq for further manipulations, in particular, append date to each input line change?

  • Of course uniq doesn't produce what you call "realtime" output. It emits only unique lines - to do that it has to process the entire set of input lines and remove duplicates. It can't do that until the input is complete, so you won't see any output until the input pipe is closed. – Andrew Henle Jul 14 '16 at 9:17
  • Form man uniq: Note: uniq does not detect repeated lines unless they are adjacent. You may want to sort the input first... – choroba Jul 14 '16 at 9:19
  • both of these comments are great but missing the point... it should be semi-realtime because it prints the uniq lines that are queued up once it finds a difference, because as both of them have said, it only looks when they're adjacent. But using sort will do the opposite... it will make it impossible to be realtime until after the sort step because sort must process everything before printing anything. – Peter Jul 14 '16 at 9:24
  • @AndrewHenle when uniq is last in piped expression, I see realtime behavior. I don't do sort or count exactly for this purpose – danbst Jul 14 '16 at 9:28
4

Perhaps you only have to change the buffering:

# buffering totally off
stdbuf -i0 -o0 -e0 command

# buffering per line only
stdbuf -oL -eL command

It works for me here... your results may vary. If it doesn't work, try putting it before different commands (not while or read though, since they are bash builtins).

for i in {1..2000}; do
    sleep 0.1 && echo $(netstat -s | grep -i syns)
done \
| stdbuf -oL -eL uniq | while read line; do
    echo $(date) $line
done
  • wow, stdbuf is exactly what I expected to exist! – danbst Jul 14 '16 at 9:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.