This may be a silly question, but I ask it still. If I have declared a shebang

#!/bin/bash 

in the beginning of my_shell_script.sh, so do I always have to invoke this script using bash

[my@comp]$bash my_shell_script.sh

or can I use e.g.

[my@comp]$sh my_shell_script.sh

and my script determines the running shell using the shebang? Is it the same happening with ksh shell? I'm using AIX.

  • 6
    there is a little confusion on your part: when you do "_some_shell some_script" it starts _some_shell and asks it to interpret some_script. So no, if you do "sh my_shell_script.sh" it won't interpret the shebang, but will interpret the script in sh instead. To use the shebang : chmod +x my_shell_script.sh ; /path/to/my_shell_script.sh # or ./my_shell_script.sh if you happen to be in its directory – Olivier Dulac Aug 21 '13 at 11:04
up vote 111 down vote accepted

The shebang #! is an human readable instance of a magic number consisting of the byte string 0x23 0x21, which is used by the exec() family of functions to determine whether the file to be executed is a script or a binary. When the shebang is present, exec() will run the executable specified after the shebang instead.

Note that this means that if you invoke a script by specifying the interpreter on the command line, as is done in both cases given in the question, exec() will execute the interpreter specified on the command line, it won't even look at the script.

So, as others have noted, if you want exec() to invoke the interpreter specified on the shebang line, the script must have the executable bit set and invoked as ./my_shell_script.sh.

The behaviour is easy to demonstrate with the following script:

#!/bin/ksh
readlink /proc/$$/exe

Explanation:

  • #!/bin/ksh defines ksh to be the interpreter.

  • $$ holds the PID of the current process.

  • /proc/pid/exe is a symlink to the executable of the process (at least on Linux; on AIX, /proc/$$/object/a.out is a link to the executable).

  • readlink will output the value of the symbolic link.

Example:

Note: I'm demonstrating this on Ubuntu, where the default shell /bin/sh is a symlink to dash i.e. /bin/dash and /bin/ksh is a symlink to /etc/alternatives/ksh, which in turn is a symlink to /bin/pdksh.

$ chmod +x getshell.sh
$ ./getshell.sh 
/bin/pdksh
$ bash getshell.sh 
/bin/bash
$ sh getshell.sh 
/bin/dash
  • thanks Thomas for this answer. Pretend we launch the script as a child process from Node.js or Java or whatever. Can we launch an "exec" process, and then exec will run the shell script? I ask beause I am looking for answers to this question: stackoverflow.com/questions/41067872/… – Alexander Mills Dec 9 '16 at 20:26
  • 1
    @AlexanderMills The exec() referred to in this answer is a system call, the command exec is a shell builtin, which is why you cannot invoke an exec program from Node.js or Java. However, any shell command invoked by e.g. Runtime.exec() in Java is eventually processed by the exec() system call. – Thomas Nyman Dec 11 '16 at 16:02
  • Huh, yeah I am indeede familiar with the Java API you just mentioned, I wonder if there is a way to invoke the lower level exec() call from Node.js somehow – Alexander Mills Dec 11 '16 at 19:51
  • @AlexanderMills I would imagine child_process.{exec(),execFile(),spawn()} all would be implemented using the C exec() (through process). – Thomas Nyman Dec 12 '16 at 3:02

Yes it does. By the way it is not a silly question. A reference for my answer is here. Starting a Script With #!

  • It is called a shebang or a "bang" line.

  • It is nothing but the absolute path to the Bash interpreter.

  • It consists of a number sign and an exclamation point character (#!), followed by the full path to the interpreter such as /bin/bash.

    All scripts under Linux execute using the interpreter specified on a first line Almost all bash scripts often begin with #!/bin/bash (assuming that Bash has been installed in /bin) This ensures that Bash will be used to interpret the script, even if it is executed under another shell. The shebang was introduced by Dennis Ritchie between Version 7 Unix and 8 at Bell Laboratories. It was then also added to the BSD line at Berkeley .

Ignoring An Interpreter Line (shebang)

If you do not specify an interpreter line, the default is usually the /bin/sh. But, it is recommended that you set #!/bin/bash line.

  • 3
    To elaborate, the kernel only knows how to execute statically linked binaries and where to find interpreter information for others (a special field in the binary, or the shebang line). Typically executing a shell script means following the shebang line to the shell, and then following the DT_INTERP field in the shell binary to the dynamic linker. – Simon Richter Aug 21 '13 at 6:37
  • 4
    Also note that this isn't limited to shell scripts. All text based script files use this. eg #!/usr/bin/perl #!/usr/local/bin/python #!/usr/local/bin/ruby Another common shebang entry used to support multiple systems is to use env to locate the interpreter you want to use, like #!/usr/bin/env perl #!/usr/bin/env python – sambler Aug 21 '13 at 7:31
  • @sambler speaking of env, which should be actually prefered? Python and Perl often use env, while on shellscripts, this is often omitted and shebang points to the shell in question. – polemon Aug 21 '13 at 10:22
  • 1
    @polemon less of which is preferred and more on which paths vary. The basic shells are in the same path on all systems. Up to date versions of perl and python can be installed in different locations on different systems so using env allows the same shebang to always work, which is why env is used more with perl and python scripts than shell scripts. – sambler Aug 22 '13 at 8:23
  • env to find a program in $PATH is a bit of a hack. It doesn't set environment variables like the name implies. $PATH might be a different result for different users. But it helps scripts run without modification on systems which put a reasonable perl interpreter in some odd place. – John Mahowald Jul 24 at 22:08

In fact, if you take it consequently, the executable noted in the shebang line, is just an executable. It makes sense to use some text interpreter as executable, but it is not necessary. Just for clarification and demonstration, i made a rather useless test:

#!/bin/cat
useless text
more useless text
still more useless text

Named the file test.txt and set the exectuable bit chmod u+x test.txt, then "called" it: ./test.txt. As expected, the contents of the file is output. In this case, cat does not ignore the shebang line. It simply outputs all lines. Any useful interpreter should thus be able to ignore this shebang line. For bash, perl and PHP, it is simply a comment line. So yes, these ignore the shebang line.

The exec system call of the Linux kernel understands shebangs (#!) natively

When you do on bash:

./something

on Linux, this calls the exec system call with the path ./something.

This line of the kernel gets called on the file passed to exec: https://github.com/torvalds/linux/blob/v4.8/fs/binfmt_script.c#L25

if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!'))

This reads the very first bytes of the file, and compares them to #!.

If that is true, then the rest of the line is parsed by the Linux kernel, which makes another exec call with path /usr/bin/env python and the current file as the first argument:

/usr/bin/env python /path/to/script.py

and this works for any scripting language that uses # as a comment character.

And yes, you can make an infinite loop with:

printf '#!/a\n' | sudo tee /a
sudo chmod +x /a
/a

Bash recognizes the error:

-bash: /a: /a: bad interpreter: Too many levels of symbolic links

#! just happens to be human readable, but that is not required.

If the file started with different bytes, then the exec system call would use a different handler. The other most important built-in handler is for ELF executable files: https://github.com/torvalds/linux/blob/v4.8/fs/binfmt_elf.c#L1305 which checks for bytes 7f 45 4c 46 (which also happen to be human readable for .ELF). This reads the ELF file, puts it into memory correctly, and starts it running. See also: https://stackoverflow.com/questions/8352535/how-does-kernel-get-an-executable-binary-file-running-under-linux/31394861#31394861

Finally, you can add your own shebang handlers with the binfmt_misc mechanism. For example, you can add a custom handler for .jar files. This mechanism even supports handlers by file extension. Another application is to transparently run executables of a different architecture with QEMU.

I don't think POSIX specifies shebangs, however: https://unix.stackexchange.com/a/346214/32558, although it does mention it in rationale sections, and in the form "if executable scripts are supported by the system something may happen".

  • 1
    Running ./something from a shell won't pass the full path to exec, but exactly the path entered. Can you correct this in your answer? Do echo "$0" in your script and you will see this is the case. – AndiDog Jul 26 at 7:46

From what I gathered, whenever a file has an executable bit set and is invoked, the kernel analyzes the file header in order to determine how to proceed (as far as I know, you can add custom handlers for custom file formats via LKMs). If the file appears to be a text file with a #! combination in the beginning, its execution is dispatched to another executable (usually a shell of sorts), a path to which is to be specified directly after the said shebang, in the same line. The kernel then proceeds to execute the shell and pass the file for it to handle.

In short, it doesn't matter which shell you invoke the script with - the kernel will dispatch the execution to the appropriate one either way.

  • 4
    There's a marked difference between bash ./myscript.sh and ./myscript.sh. – a CVn Aug 21 '13 at 9:35
  • What do you mean by this "marked difference"? – jrara Aug 21 '13 at 9:59
  • 3
    @jrara See my answer, the statement that "it doesn't matter which shell you invoke the script with" is simply not true. – Thomas Nyman Aug 21 '13 at 10:33

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.