Some sed commands seem to fail when run in background. Minimal example below.

One sed command appends append after a block of lines that begin with a star *. The other just substitutes a word. The order of the sed commands does not matter.

TEXT=$'* first\n'

two_seds(){ echo "$TEXT" | \
    sed "$!N;/.*\n\*/! s/\n\(.*\)/\nappend\n\1/;P;D" | \
    sed "s/first/second/";}

Executing two_seds & wait results to:

* second

While executing wait & two_seds yields:

* second

So the second sed command always works, while the first does nothing after an &.

While for now I know how to work around it, I would love to understand what's going on. Partly out of curiosity, partly so that it won't bite me later in obscure edge cases.

Can reproduce in bash or dash. Running:

Ubuntu 18.04
sed (GNU sed) 4.4
GNU bash, version 4.4.19(1)-release (x86_64-pc-linux-gnu)
dash, version 0.5.8-2.10
  • 2
    Is it your intention to use $! as the last backgrounded-PID (since it's in double quotes), or as an address range "not the last line"? – Jeff Schaller Nov 27 '18 at 18:08
  • The intention was to have it as "not the last line". The use of double quotes was because there are other variables there that need to be resolved. Apparently "not the last line" is not needed for my use case, so I can drop it. – Boffin Nov 28 '18 at 8:58
  • Do you still see the problem after you drop the $! ? – JigglyNaga Nov 28 '18 at 10:26
  • @JigglyNaga Nope, that solves it. It didn't do anything constructive anyways, since it's resolved to an empty string in the "working" case. – Boffin Nov 28 '18 at 19:35

The "$!" at the beginning of your sed script ("$!N;/.*\n\...) will expand to the PID of the last background job. Shell variables are expanded inside double quotes.

If no background job was yet run, it will expand to nothing, which is wrong, too (if you put the sed script in single quotes, it will go in an infinite loop).

  • Thanks a lot for solving this mystery @pizdelect. Much to learn, I have. – Boffin Nov 28 '18 at 9:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.