I was writing a bash script, and happened to updated the code (saved the script file to disk) while the script was waiting for some input in a while loop. After I returned to the terminal and continued with the previous invocation of the script, bash gave an error about the file syntax:

/home/aularon/bin/script: line 58: unexpected EOF while looking for matching `"'
/home/aularon/bin/script: line 67: syntax error: unexpected end of file

So I tried doing the following:

1st: create a script, self-update.sh let's call it:

cat $0 | sed 's/BEFORE\./AFTER!./' > $fname
cp $fname $0
rm -f $fname
echo 'String: BEFORE.';

What the script does is reading its code, changing the word 'BEFORE' into 'AFTER', then re-write itself with the new code.

2nd Run it:

chmod +x self-update.sh

3rd wonder...

aularon@aularon-laptop:~$ ./self-update.sh 
String: AFTER!.

Now, I wouldn't have guessed that on the same invocation it would output AFTER!, on the second run for sure, but not on the first one.

So my question is: is it intentional (by design)? or it is because of the way bash runs the script? Line by line or command by command. Is there any good use of such behaviour? Any example of it?

Edit: I tried reformatting the file to put all command in one line, it does not work now:

fname=$(mktemp);cat $0 | sed 's/BEFORE\./AFTER!./' > $fname;cp $fname $0;rm -f $fname;echo 'String: BEFORE.';


aularon@aularon-laptop:~$ ./self-update.sh #First invocation
String: BEFORE.
aularon@aularon-laptop:~$ ./self-update.sh #Second invocation
String: AFTER!.

While when moving the echo string to the next line, separating it from the rewriting (cp) call:

fname=$(mktemp);cat $0 | sed 's/BEFORE\./AFTER!./' > $fname;cp $fname $0;rm -f $fname;
echo 'String: BEFORE.';

And now it works again:

aularon@aularon-laptop:~$ ./self-update.sh 
String: AFTER!.
  • 1
  • 1
    I've looked through the source, but I can't really find any explanation for this. I'm pretty sure though that I've seen some self-extracting installers that were implemented as shell scripts a few years ago. These scripts contains some lines of executable shell commands, and then a huge data block that is read by these commands. So I'll assume that this is designed this way so that bash doesn't have to read the whole script into memory which wouldn't work with huge self-extracting scripts. – Martin von Wittich Jan 23 '14 at 21:42
  • Here's an example of an SFX script: ftp.games.skynet.be/pub/wolfenstein/… – Martin von Wittich Jan 23 '14 at 21:43
  • Nice! I remember seeing these self installing scripts, AMD Catalyst driver (proprietary) still ships like this, it self-extracts and then installs. These depend on such behaviour of reading the files in chunks. Thanks for the example! – aularon Jan 23 '14 at 21:53

This is by design. Bash reads scripts in chunks. So it'll read a portion of the script, run any lines that it can, and then read the next chunk.

So you run into something like this:

  • Bash reads the first 256 bytes (bytes 0-255) of the script.
  • Within that first 256 bytes is a command that takes a while to run, and bash starts that command, waiting for it to exit.
  • While the command is running, the script is updated, and the portion changed is after the 256 bytes it already read.
  • When the command bash was running finishes, it continues reading the file, resuming from where it was, getting bytes 256-511.
  • That portion of the script changed, but bash doesn't know that.

Where this becomes even more problematic is that if you edit anything before byte 256. Lets say you delete a couple of lines. Then the data in the script that was at byte 256, is now somewhere else, say at byte 156 (100 bytes earlier). Because of this, when bash continues reading, it's going to get what was originally 356.

This is just an example. Bash doesn't necessarily read 256 bytes at a time. I don't know exactly how much it reads at a time, but it doesn't matter, behavior is still the same.

|improve this answer|||||
  • No, it reads by chunks, but rewinds to where it was to be sure to read the next command as it is after the previous command has returned. – Stéphane Chazelas Jan 23 '14 at 23:55
  • @StephaneChazelas Where are you getting that from? I just did an strace and it doesn't so much as even stat the file to see if it's changed. No lseek calls. – Patrick Jan 24 '14 at 1:25
  • See pastie.org/8662761 for the relevant parts of the strace output. See how the echo foo changed to a echo bar during the sleep. It's been behaving like that for as far back as versions 2, so I don't think it's a version issue. – Stéphane Chazelas Jan 24 '14 at 7:35
  • Apparently it reads the file line by line, I tried with the file and that what I found out. I will edit my question to highlight that behaviour. – aularon Jan 24 '14 at 11:52
  • @Patrick if you can update your answer to reflect it is reading line by line, so I can accept your answer. (Check my edit to the question on that matter). – aularon Jan 24 '14 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.