Am I hitting a race condition in bash?

Question

I have a script which compares the output of a command with the output of the same command as it was ran previously, it works most of the time, but every now and then it doesn't work as expected.

I've been able to reproduce the issue in one test line. I know I could easily break this out into comparing two separate files and the problem would go away, but I'd like to understand what's actually happening here and if there's a way to achieve what I'm trying to achieve in the way I'm trying to achieve it.

Below is the output of my command ran several times, you can see the that it echoes "test" in one of the cases, but most of the time it works as expected.

root@dev:~# comm -13 /tmp/test <(echo '"test"' | cut -d'"' -f2 | sort -u | tee /tmp/test)
root@dev:~# comm -13 /tmp/test <(echo '"test"' | cut -d'"' -f2 | sort -u | tee /tmp/test)
root@dev:~# comm -13 /tmp/test <(echo '"test"' | cut -d'"' -f2 | sort -u | tee /tmp/test)
root@dev:~# comm -13 /tmp/test <(echo '"test"' | cut -d'"' -f2 | sort -u | tee /tmp/test)
root@dev:~# comm -13 /tmp/test <(echo '"test"' | cut -d'"' -f2 | sort -u | tee /tmp/test)
test
root@dev:~# comm -13 /tmp/test <(echo '"test"' | cut -d'"' -f2 | sort -u | tee /tmp/test)
root@dev:~# comm -13 /tmp/test <(echo '"test"' | cut -d'"' -f2 | sort -u | tee /tmp/test)
root@dev:~# comm -13 /tmp/test <(echo '"test"' | cut -d'"' -f2 | sort -u | tee /tmp/test)

I'm running Ubuntu 10.04, bash 4.1-2ubuntu3.5 and coreutils 7.4-2ubuntu3

Answer 1

Yes, that's a race condition.

The problem is that the shell starts all processes in the pipeline at the same time and tee truncates the output file on startup. If tee is faster then comm the file is empty for comm otherwise it is not.

The pipeline behaviour can be seen if you run this several times (mabe in a loop):

date '+first:  %T'|(cat>&2;sleep 5)|date '+second: %T'

Answer 2

Your line confused me a bit as it was not clear what both occurrences of "test" are supposed to be. - Therefore I really understand that your shell is confused, too. ;)

If I understand correctly the first "/tmp/test" should correspond with the old output of the command and the second one corresponds with the new output.

You can be sure that stdin will contain the new output, but the content of the file is undefined: At the point comm reads that file, tee might not yet be started. In that case it will still contain the old "test" line from the last run. Or tee might already have finished. In that case it will contain the "test" line of the current run. Or tee just started, already cleared the file and is about to write the new content. If that happens you will compare the "test" from stdin with the empty file and you will see your output.

You have two different outputs you want to compare. I do not see a way to do this without having two different files. Having two different (properly named) files also makes the line much more readable and solves the confusion at the beginning. ;)