13

I was writing a bash script, and happened to updated the code (saved the script file to disk) while the script was waiting for some input in a while loop. After I returned to the terminal and continued with the previous invocation of the script, bash gave an error about the file syntax:

/home/aularon/bin/script: line 58: unexpected EOF while looking for matching `"'
/home/aularon/bin/script: line 67: syntax error: unexpected end of file

So I tried doing the following:

1st: create a script, self-update.sh let's call it:

#!/bin/bash
fname=$(mktemp)
cat $0 | sed 's/BEFORE\./AFTER!./' > $fname
cp $fname $0
rm -f $fname
echo 'String: BEFORE.';

What the script does is reading its code, changing the word 'BEFORE' into 'AFTER', then re-write itself with the new code.

2nd Run it:

chmod +x self-update.sh
./self-update.sh

3rd wonder...

aularon@aularon-laptop:~$ ./self-update.sh 
String: AFTER!.

Now, I wouldn't have guessed that on the same invocation it would output AFTER!, on the second run for sure, but not on the first one.

So my question is: is it intentional (by design)? or it is because of the way bash runs the script? Line by line or command by command. Is there any good use of such behaviour? Any example of it?


Edit: I tried reformatting the file to put all command in one line, it does not work now:

#!/bin/bash
fname=$(mktemp);cat $0 | sed 's/BEFORE\./AFTER!./' > $fname;cp $fname $0;rm -f $fname;echo 'String: BEFORE.';

Output:

aularon@aularon-laptop:~$ ./self-update.sh #First invocation
String: BEFORE.
aularon@aularon-laptop:~$ ./self-update.sh #Second invocation
String: AFTER!.

While when moving the echo string to the next line, separating it from the rewriting (cp) call:

#!/bin/bash
fname=$(mktemp);cat $0 | sed 's/BEFORE\./AFTER!./' > $fname;cp $fname $0;rm -f $fname;
echo 'String: BEFORE.';

And now it works again:

aularon@aularon-laptop:~$ ./self-update.sh 
String: AFTER!.
4
  • 1
    Related: stackoverflow.com/questions/4754592/… Jan 23, 2014 at 21:10
  • 1
    I've looked through the source, but I can't really find any explanation for this. I'm pretty sure though that I've seen some self-extracting installers that were implemented as shell scripts a few years ago. These scripts contains some lines of executable shell commands, and then a huge data block that is read by these commands. So I'll assume that this is designed this way so that bash doesn't have to read the whole script into memory which wouldn't work with huge self-extracting scripts. Jan 23, 2014 at 21:42
  • Here's an example of an SFX script: ftp.games.skynet.be/pub/wolfenstein/… Jan 23, 2014 at 21:43
  • Nice! I remember seeing these self installing scripts, AMD Catalyst driver (proprietary) still ships like this, it self-extracts and then installs. These depend on such behaviour of reading the files in chunks. Thanks for the example!
    – aularon
    Jan 23, 2014 at 21:53

1 Answer 1

16

This is by design. Bash reads scripts in chunks. So it'll read a portion of the script, run any lines that it can, and then read the next chunk.

So you run into something like this:

  • Bash reads the first 256 bytes (bytes 0-255) of the script.
  • Within that first 256 bytes is a command that takes a while to run, and bash starts that command, waiting for it to exit.
  • While the command is running, the script is updated, and the portion changed is after the 256 bytes it already read.
  • When the command bash was running finishes, it continues reading the file, resuming from where it was, getting bytes 256-511.
  • That portion of the script changed, but bash doesn't know that.

Where this becomes even more problematic is that if you edit anything before byte 256. Lets say you delete a couple of lines. Then the data in the script that was at byte 256, is now somewhere else, say at byte 156 (100 bytes earlier). Because of this, when bash continues reading, it's going to get what was originally 356.

This is just an example. Bash doesn't necessarily read 256 bytes at a time. I don't know exactly how much it reads at a time, but it doesn't matter, behavior is still the same.

7
  • 1
    No, it reads by chunks, but rewinds to where it was to be sure to read the next command as it is after the previous command has returned. Jan 23, 2014 at 23:55
  • 1
    @StephaneChazelas Where are you getting that from? I just did an strace and it doesn't so much as even stat the file to see if it's changed. No lseek calls.
    – phemmer
    Jan 24, 2014 at 1:25
  • 1
    See pastie.org/8662761 for the relevant parts of the strace output. See how the echo foo changed to a echo bar during the sleep. It's been behaving like that for as far back as versions 2, so I don't think it's a version issue. Jan 24, 2014 at 7:35
  • 1
    Apparently it reads the file line by line, I tried with the file and that what I found out. I will edit my question to highlight that behaviour.
    – aularon
    Jan 24, 2014 at 11:52
  • 1
    @Patrick if you can update your answer to reflect it is reading line by line, so I can accept your answer. (Check my edit to the question on that matter).
    – aularon
    Jan 24, 2014 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.