2

awk behaves differently when analyzing the output of tail -f versus inotifywait -m. Specifically I am searching for a matching string and want to exit awk once it appears. This works fine for tail -f, however in the case of inofitywait, awk needs to be triggered twice. Why so?


Reproducible Examples:

Say we are searching for a specific string ("OPEN") in either case and use a special exit code as marker. Let's also trigger it after a short wait (inotofywait needs a moment) and return the exit code. Pseudo-code:

 command | awk-analysis || get-non-0-exit-code & trigger

All good for tail -f

The following a) prints the line, b) returns the exit code and c) terminates. As expected.

tail -f test.file | awk '/OPEN/ { print $0 ; exit 100 }' || echo $? &
  { sleep 2 ; echo OPEN > test.file ; }

However with inotifywait -m

the result is quite different.

inotifywait -m -e OPEN | awk '/OPEN/ { print $0 ; exit 100 }' || echo $? &
  { sleep 2 ; touch test.file ; }

This will print the line (so inotifywait is triggered and awk sees it) but NOT show the exit code nor terminate. Only another trigger like touch test.file is able to stop awk

Maybe control characters?

I thought maybe I am missing a singal awk uses here, so I tried to analyze with cat -A (results file in parent folder, so otherwise a second "OPEN" is triggered in inotifywait):

tail -f test.file | tee >(cat -A >../stream) | ....
cat ../stream
OPEN$

and

inotifywait -m -e OPEN | tee >(cat -A >../stream) | ....
cat ../stream
./ OPEN test.file$

So no unseen control characters missed.

What is the reason for this behaviour?

Am I missing a newline? How comes awk does print the line, but not run the exit command in the same code block? Why does it work with tail, though?


Verisons

awk --verison : GNU Awk 4.2.1, API: 2.0 (GNU MPFR 4.0.2, GNU MP 6.1.2)

inotifywait -h : inotifywait 3.14

tail --verison : tail (GNU coreutils) 8.30


EDIT due to Kusalananda's comments:

tail -f

test.file exists and has the following content:

*case 1

OPEN
spam

*case 2

spam
OPEN

*case 3 (file is empty)

Trigger as above is NOT run, i.e.

tail -f test.file | awk '/OPEN/ {print $0 ; exit 100 }' || echo $?

Case 1 & 2 : immediately returns the matching line, exit code and is terminated.

Case 3: waits, open other terminal and echo OPEN >> 3 or echo OPEN > 3 returns string, exit code and terminates.

7
  • With the tail -f example, is the line containing OPEN the last line in the file? If not, try again with OPEN only on the last line of that file. Also try with appending OPEN to the file instead of truncating it. In either case, the left hand side would not terminate immediately just because awk exits, it would need to try (and fail) to write across the pipe. – Kusalananda Jan 31 at 21:46
  • 2
    My answer here explains why that only works with GNU tail, what extra stuff GNU tail does for it to work, and how you can make it work with any tail implementation (which would work with inotifywait too). – user414777 Jan 31 at 21:52
  • @Kusalananda see edit. – FelixJN Jan 31 at 22:01
  • @Fiximan I think Kamil is on the right track in their answer, assuming you're using GNU tools. – Kusalananda Jan 31 at 22:03
  • @Kusalananda Indeed. – FelixJN Jan 31 at 22:22
2

echo $? runs after the entire pipeline (the code before ||) terminates.

In the case of inotifywait … | awk … || echo … after awk terminates inotifywait still runs. It will get SIGPIPE only if (when) it tries to write more. Try touch test.file again to get to this point and trigger echo.

On the other hand tail in tail -f … | awk … terminates immediately after awk exits because GNU tail takes special steps to detect this situation.


In order to reproduce this with inotifywait, one would need to forward the PID and send a SIGPIPE via awk:

{ inotifywait  -m -e OPEN ./  & echo $! ; } |
awk 'NR==1 {pid=$0}
     /OPEN/ {print $0,pid
             system("echo kill -13 "pid)
             exit 122  }' ||
echo $? 
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.