15

Generally, shell scripts contain the following comment at the first line of the script file: #!/bin/sh. According to the researches that I made, this is called "hash bang" and it is conventional comment. This comment informs Unix that this file is executed by the Bourne Shell under the directory /bin.

My question begins in that point. Up to now I have not seen this comment like #!/bin/bash. It is always #!/bin/sh. However, Ubuntu distributions do not have the Bourne Shell program. They have the Bourne Again Shell (bash).

In that point, is it correct to place the comment #!/bin/sh in shell scripts written in Ubuntu distributions?

16

#!/bin/sh should work on all Unix and Unix-like distributions. It is generally thought of as the most portable hashbang so long as your script is kept POSIX compliant. The /bin/sh shell is supposed to be a shell that implements the POSIX shell standard, regardless of what the actual shell is that masquerade as the /bin/sh shell.


#!/bin/sh is normally just a link now as the Bourne shell is no longer maintained. On many Unix systems /bin/sh will be a link to /bin/ksh or /bin/ash, on many RHEL based Linux systems it will be a link to /bin/bash, however on Ubuntu and many Debian based systems it is a link to /bin/dash. All shells, when invoked as sh, will enter POSIX compatibility mode.

The hashbang is an important placeholder though because it allows for much greater portability than other methods, so long as your script is strictly POSIX compliant (repeated to stress importance).

Note: When bash is invoked in POSIX mode it will still allow some non-POSIX things like [[, arrays, and more. Those things may fail on a non-bash system.

  • 3
    "on Ubuntu it is a link to /bin/dash" and in general other Debian-based systems. IIRC on FreeBSD/GhostBSD it's a symlink to /bin/ash as well. – Sergiy Kolodyazhnyy Jan 25 at 0:14
  • To be fully portable, put a space between the shebang and the (absolute) interpreter path. Some systems look for #! / instead of just #!. – Simon Richter Jan 25 at 13:40
  • 5
    @SimonRichter A reference for this would be nice to see. – Kusalananda Jan 25 at 13:50
  • @SergiyKolodyazhnyy The installed version of sh on FreeBSD is ash and not a symlink. – Rob Jan 25 at 14:18
  • 2
    @Kusalananda, hmm, this page says that this is a myth, so it can probably be disregarded. – Simon Richter Jan 25 at 14:22
12

You've got it backward. /bin/sh is almost never a Bourne shell these days, and it's when it is¹ that you have a problem when you use a #! /bin/sh she-bang.

The Bourne shell was a shell written in the late 70s and replaced the previous Thompson shell (also called sh). In the early 80s, David Korn wrote a few extensions for the Bourne shell, fixed a few of the bugs and design awkwardnesses (and introduced some) and called it the Korn shell.

In the early 90s, POSIX specified the sh language based on a subset of the Korn shell and most¹ systems have now changed their /bin/sh to either the Korn shell or a shell compliant to that specification. In the case of BSDs, they gradually changed their /bin/sh, initially (after they could no longer use the Bourne shell for license reasons) the Almquist shell, a clone of the Bourne shell with some ksh extensions so it became POSIX compliant.

Today, all POSIX systems have a shell called sh (most often, but not necessarily in /bin, POSIX doesn't specify the path of the utilities it specifies) that is mostly POSIX compliant. It is generally based on either ksh88, ksh93, pdksh, bash, ash or zsh², but not the Bourne shell as the Bourne shell was never POSIX compliant³. A few of those shells (bash, zsh, yash and some pdksh derivatives enable a POSIX-compliant mode when invoked as sh and are less compliant otherwise).

bash (the GNU answer to the Korn shell) is actually the only open source shell (and one could say only currently maintained as the other ones are generally based on ksh88 which hasn't gotten any new feature since the 90s) that has been certified as being a POSIX compliant sh (as part of the macOS certification).

When you write a script with a #! /bin/sh - she-bang, you should use a standard sh syntax (and should also use standard syntax for the utilities used in that script if you want to be portable, it's not only the shell that is involved when interpreting a shell script), then it doesn't matter what implementation of that standard sh syntax interpreter is used (ksh, bash...).

It doesn't matter that those shells have extensions over the standard as long as you don't use them. It's like for writing C code, as long as you write standard C code and don't use extensions of one compiler (like gcc) or the other, your code should compile OK regardless of the compiler implementation provided that compiler is compliant.

Here, with your #! /bin/sh she-bang, your main problem would be the systems where /bin/sh is the Bourne shell that for instance doesn't support standard features like $((1+1)), $(cmd), ${var#pattern}... In which case you may need work-arounds like:

#! /bin/sh -
if false ^ true; then
  # in the Bourne shell, ^ is an alias for |, so false ^ true returns
  # true in the Bourne shell and the Bourne shell only.
  # Assume the system is Solaris 10 (older versions are no longer maintained)
  # You would need to adapt if you need to support some other system
  # where /bin/sh is the Bourne shell.
  # We also need to fix $PATH as the other utilities in /bin are
  # also ancient and not POSIX compatible.
  PATH=`/usr/xpg6/bin/getconf PATH`${PATH:+:}$PATH || exit
  export PATH
  exec /usr/xpg4/bin/sh "$0" "$@"
  # that should give us an environment that is mostly  compliant
  # to the Single UNIX Specification version 3 (POSIX.2004), the
  # latest supported by Solaris 10.
fi
# rest of the script

By the way, Ubuntu's /bin/sh is not bash by default. It's dash these days, a shell that is based on NetBSD sh, itself based on the Almquist shell which is mostly POSIX compliant except that it doesn't support multi-byte characters. On Ubuntu and other Debian-based systems, you can choose between bash and dash for /bin/sh with dpkg-reconfigure dash)4. sh scripts shipped with Debian should work the same in both shells as they are written to the Debian policy standard (a superset of the POSIX standard). You'll probably find they also work OK in zsh's sh emulation or bosh (probably not ksh93 nor yash which don't have a local builtin (required by the Debian policy but not POSIX)).

All systems in scope on unix.stackexchange.com have a POSIX sh somewhere. Most of them have a /bin/sh (you might find very rare ones that don't have a /bin directory but you probably don't care about that), and that's generally a POSIX sh interpreter (and in rare cases a (non-standard) Bourne shell instead).

But sh is the only shell interpreter executable you can be sure to find on a system. For the other shells, you can be sure macOS, Cygwin and most GNU/Linux distributions will have bash. SYSV-derived OSes (Solaris, AIX...) will generally have ksh88, possibly ksh93. OpenBSD, MirOS will have a pdksh derivative. macOS will have zsh. But out of that, there will be no guarantee. No guarantee either of whether bash or any of those other shells will be installed in /bin or elsewhere (it's usually found in /usr/local/bin on BSDs when installed for instance). And of course no guarantee of the version of the shell that will be installed.

Note that the #! /path/to/executable is not a convention, it's a feature of all Unix-like kernels (introduced in the early 80s by Dennis Ritchie) that allows executing arbitrary files by specifying the path of the interpreter in a first line starting with #!. It can be any executable.

When you execute a file whose first line starts with #! /some/file some-optional-arg, the kernel ends up executing /some/file with some-optional-arg, the path of the script and the original arguments as arguments. You can make that first line #! /bin/echo test to see what's happening:

$ ./myscript foo
test ./myscript foo

When you use /bin/sh - instead of /bin/echo test, the kernel executes /bin/sh - ./myscript foo, sh interprets the content code stored in myscript and ignores that first line as it's a comment (starts with #).


¹ Probably the only system today where any of us will come across a /bin/sh that is based on the Bourne shell is Solaris 10. Solaris is one of the few Unices that decided to keep a Bourne shell there for backward compatibility (the POSIX sh language is not fully backward compatible with the Bourne shell) and (at least for the desktop and full server deployments) have the POSIX sh elsewhere (in /usr/xpg4/bin/sh, based on ksh88), but that changed in Solaris 11 where /bin/sh is now ksh93. The other ones are mostly defunct.

² The /bin/sh of MacOS/X used to be zsh, but later changed to bash. It's not zsh's primary focus to be used as a POSIX sh implementation. Its sh mode is primarily to be able to embed or call (source) POSIX sh code in zsh scripts

³ Recently, @schily extended the OpenSolaris shell (base on the SVR4 shell, based on the Bourne shell) to become POSIX compliant, called bosh but I'm not aware that it's used on any system yet. Along with ksh88 that makes it a second POSIX-compliant shell based on the code of the Bourne shell

4 In older versions, you could also use mksh or its more POSIX lksh incarnation. That's the the MirOS (formerly MirBSD) shell based on pdksh, itself based on the Forsyth shell (another reimplementation of the Bourne shell))

  • Minor thing (about the shell code mid-answer): Should you not use just exec sh "$0" "$@" after setting the PATH? That should pick sh up from the correct place, I would have thought. – Kusalananda Jan 25 at 13:52
  • 1
    @Kusalananda, that should work but is more risky as we could end up in an endless loop if something's amiss (like sh with wrong permissions, getconf modified...). Anyway, the rest is specific to Solaris 10, so we could also hardcode everything including the content of $PATH. – Stéphane Chazelas Jan 25 at 13:57
  • Any citation for OSX using ksh as its POSIX shell. Its default shell used to be tcsh but I don't remember bash not being used especially as Korn Shell was not open source until after OSX came out – Mark Jan 25 at 14:19
  • 1
    @Mark, I didn't say OSX ever used ksh. I did say it used to be zsh and they switched to bash (a special build of bash). – Stéphane Chazelas Jan 25 at 14:24
  • 1
    @fpmurphy, if you look at the early versions, bash was already siding with ksh wherever ksh was not compatible with the Bourne shell. While you had to wait until bash2 to get to the same feature-level as ksh88, bash initially already had several of ksh's extensions like $(...), emacs/vi modes, tilde/brace expansions (those one initially from csh), fc, typeset, alias, ksh-style function syntax... – Stéphane Chazelas Jan 25 at 16:32
8

Yes you can use #!/bin/sh in a script because /bin/sh is (hopefully) provided for on such systems, usually via a link of some sort that makes bash behave (more or less) like a sh would. Here's a Centos7 system, for example, that links sh to bash:

-bash-4.2$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Dec  4 16:48 /bin/sh -> bash
-bash-4.2$ 

You could also use #!/bin/bash if you are writing a bash script only for that system and want to use bash features. However, such scripts will suffer from portability problems, for example on OpenBSD where bash is only installed if the admin takes the trouble to install it (I do not) and then it is installed to /usr/local/bin/bash, not /bin/bash. A strictly POSIX #!/bin/sh script should be more portable.

  • Note this: "When invoked as sh, Bash enters POSIX mode after reading the startup files." -- gnu.org/software/bash/manual/bashref.html#Bash-POSIX-Mode – glenn jackman Jan 24 at 18:46
  • 2
    When I write scripts for RHEL servers at my job, I use #!/bin/bash precisely so that I can take advantage of non-POSIX features. – Monty Harder Jan 24 at 19:46
  • @MontyHarder IIRC on RHEL /bin/sh it's a symlink to bash, is that correct ? I know for sure on CentOS it is, though. – Sergiy Kolodyazhnyy Jan 25 at 0:10
  • 1
    @SergiyKolodyazhnyy Yes, on the RHEL server I just checked, it's a symlink to bash. The binary knows what name was used to invoke it, and when it's called sh it acts like old-school Bourne sh. – Monty Harder Jan 28 at 20:02
6

You asked

is it correct to place the comment #!/bin/sh in shell scripts written in ubuntu distributions ?

The answer depends on what you write in the shell script.

  • If you strictly use portable POSIX-compliant scripts, and don't use any bash-specific commands, then you can use /bin/sh.

  • If you know that you are only ever using the script on a machine with bash, and you want to use bash-specific syntax, then you should use /bin/bash

  • If you're want to be sure that the script will work on an assortment of unix machines, then you should use only POSIX-compliant syntax, and /bin/sh

  • If you regularly use another shell (e.g. ksh, zsh or tcsh), and want to use that syntax in your script, then you should use the appropriate interpreter (like /bin/ksh93, /bin/zsh, or /bin/tcsh)

3

The "#!" comment does not always use /bin/bash or /bin/sh. It just lists whatever the interpreter should be, not just for shell scripting. For example my python scripts usually start with #!/usr/bin/env python.

Now the difference between #!/bin/sh and #!/bin/bash is that /bin/sh is not always a symlink to /bin/bash. Often but not always. Ubuntu is a notable exception here. I have seen scripts working fine on CentOS but failing on Ubuntu because author used bash-specific syntax with #!/bin/sh.

1

Sorry for pouring some cold water on all those great answers that say that /bin/sh is present in all Unix systems -- it is present, except in the most used Unix system of all times: Android.

Android has its shell in /system/bin/sh, and there's usually no way to make a /bin/sh link, even on a rooted system (because of the way the system is locked down by use of selinux and capabilities(7) bounding sets).

For those who will say that android is not POSIX-conformant: neither are most Linux and BSD distributions. And the existence of /bin/sh with this path is not mandated by the standard:

Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh, and should be determined by interrogation of the PATH returned by getconf PATH, ensuring that the returned pathname is an absolute pathname and not a shell built-in.

  • While all systems that try to be POSIX compliant have numerous conformance bugs (including certified ones), Android (and most other embedded systems like your router or lightbulb OS) don't intend to be POSIX compliant. Android is off-topic here except when it comes down to its POSIX interface. – Stéphane Chazelas Jan 25 at 14:01
  • @Christopher, which getconf? For instance on Solaris 11.4, would that be the one in /usr/bin? The one in /usr/xpg4/bin (for SUSv2 compliance), the one in /usr/xpg6/bin (for SUSv3 compliance)? /usr/xpg7/bin (for SUSv4)? – Stéphane Chazelas Jan 25 at 14:07
  • @StéphaneChazelas if we're up to judging intentions, I think I can easily dig out some Linus Torvalds or Theo de Raadt quote to the effect that they don't care that much about POSIX ;-) And most embedded systems are based on linux + busybox, which is hardly less POSIX-conformant than the typical Unix-like system. And a) android is not really an "embedded" system and b) an android system could be made POSIX-compliant without having a shell in /bin/sh – mosvy Jan 25 at 14:14
  • 1
    @mosvy, what you say is mostly true. But Torvalds can only speak for the Linux kernel. Chet Ramey cares about POSIX compliance for bash, RedHat cares about POSIX compliance (they sponsor the Austin group and sit on their group meetings, they maintain a lot of the GNU software). The idea is that POSIX is the only standard that most Unix-like systems look at. Users care about POSIX compliant as it makes it easier to have software portable to different systems. It's not ideal but it's better than nothing. Android has its own programming API, the POSIX is not really a concern there. – Stéphane Chazelas Jan 25 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.