why do background jobs hang depending on the size of the output?

Question

I had some stress-testing scripts that were running in parallel and they would hang after finishing and would wait for a RETURN keystroke to exit. After investigating I discovered that it is not peculiar to my scripts but to all sorts of scripts ran on bash and that it depends on the size of the output produced (at least in my system: Ubuntu precise)

For instance the following:

find . &

hangs and waits for a RETURN keystroke if sufficient output is produced (try .. or ../.. to get more output), otherwise (i.e. if "little" output is produced) exits without hanging.

Since I find this kind of feature annoying in my particular case is there a way around it?

Answer 1

The command does not hang. You think that the command is hanging because you don't see the prompt. The prompt is there. You don't see the prompt because it was pushed up by the output of the background process. Pressing enter after the long output of a background process causes the shell to "execute" the empty line and print a new prompt.

Try the following to convince yourself:

1. execute find . &
2. wait until output done
3. see blinking cursor or something but no prompt
4. type echo foo
5. press enter
6. see foo printed and a new prompt

More experiments:

seq 10 &

this will print the numbers 1 to 10 and then print a prompt.

seq 10000 &

this will print the numbers 1 to 10000 and then you have blinking cursor and no prompt. But the prompt is there. try echo foo and press enter and your will see foo printed and a new prompt.

(sleep 2; seq 10) &

this command emulates the waiting time of a command with long output but does not have a long output. On my system this has the following effect: first sleep 2 is executed in the background. moments later the shell prints the prompt. then, after 2 seconds, seq 10 is executed in the background. this will print ten lines and push the prompt up. then the background job is done.

So you see that the background job always finishes and you always have a prompt, you just don't always see the prompt. When the background job is done quickly then the shell prints the prompt at the end and you see the prompt. When the background job takes a while printing it's output then the shell has already printed a prompt but that prompt gets pushed up so you don't see it anymore.

Even more experiments:

Try seq 10000 & or any other large number where you don't see a prompt at the end of the ouput. Now try half that number. In this example seq 5000 &. Do you see a prompt? If you do then try a larger number. For example seq 7500 &. If you don't see a prompt then try a smaller number. For example seq 2500 &. Keep doing this until you have number where you see the prompt pushed just a few lines up. The number will vary from run to run because what we have here is practically a race condition between the background process and the shell process.