18

I have this shell script saved in a file: it does some basic string substitution.

#!/bin/sh
html_file=$1
echo "html_file = $html_file"
substr=.pdf
pdf_file="${html_file/.html/.pdf}"
echo "pdf_file = $pdf_file"

If I paste it into the command line, it works fine:

$ html_file="/home/max/for_pauld/test_no_base64.html"
  echo "html_file = $html_file"
  substr=.pdf
  pdf_file="${html_file/.html/.pdf}"
  echo "pdf_file = $pdf_file"

gives

html_file = /home/max/for_pauld/test_no_base64.html
pdf_file = /home/max/for_pauld/test_no_base64.pdf

That's the output from echo above - it's working as intended.

But, when I call the script, with

$ saucer "/home/max/for_pauld/test_no_base64.html"

I get this output:

html_file = /home/max/for_pauld/test_no_base64.html
/home/max/bin/saucer: 5: /home/max/bin/saucer: Bad substitution

Is my script using a different version of bash or something? Do I need to change my shebang line?

  • 6
    That parameter expansion is a bashism (well, non-POSIX) : change your shebang. – jasonwryan Jul 19 '17 at 7:44
  • Thanks! That worked. I think I'm a bit hazy on the difference between sh and bash. I'll read up on it. If you can bothered to make your comment into an answer then I will mark it correct. – Max Williams Jul 19 '17 at 7:53
  • Gave the tick to The Duke for his detailed answer, but thanks anyway. – Max Williams Jul 19 '17 at 8:33
  • 2
    @MaxWilliams: in a nutshell, sh is the original bourne shell. All compatible scripts should be written for sh. But many subsequent shells (ex: bash, Bourne Again shell) are "almost compatible" and add a lot of extra niceness. Such as the substitution you used. Nowadays, you are quite safe using only the posix features, but be aware that portability depends on even a narrower set (ie, sh only). So in general, use: #!/usr/bin/env bash as your shebang, and use posixly defined substitutions as you wish, but with portability caveats. And read: unix.stackexchange.com/a/48787/27616 – Olivier Dulac Jul 19 '17 at 12:24
36

What is sh

sh (or the Shell Command Language) is a programming language described by the POSIX standard. It has many implementations (ksh88, dash, ...). bash can also be considered an implementation of sh (see below).

Because sh is a specification, not an implementation, /bin/sh is a symlink (or a hard link) to an actual implementation on most POSIX systems.

What is bash

bash started as an sh-compatible implementation (although it predates the POSIX standard by a few years), but as time passed it has acquired many extensions. Many of these extensions may change the behavior of valid POSIX shell scripts, so by itself bash is not a valid POSIX shell. Rather, it is a dialect of the POSIX shell language.

bash supports a --posix switch, which makes it more POSIX-compliant. It also tries to mimic POSIX if invoked as sh.

sh = bash?

For a long time, /bin/sh used to point to /bin/bash on most GNU/Linux systems. As a result, it had almost become safe to ignore the difference between the two. But that started to change recently.

Some popular examples of systems where /bin/sh does not point to /bin/bash (and on some of which /bin/bash may not even exist) are:

  1. Modern Debian and Ubuntu systems, which symlink sh to dash by default;
  2. Busybox, which is usually run during the Linux system boot time as part of initramfs. It uses the ash shell implementation.
  3. BSDs, and in general any non-Linux systems. OpenBSD uses pdksh, a descendant of the Korn shell. FreeBSD's sh is a descendant of the original UNIX Bourne shell. Solaris has its own sh which for a long time was not POSIX-compliant; a free implementation is available from the Heirloom project.

How can you find out what /bin/sh points to on your system?

The complication is that /bin/sh could be a symbolic link or a hard link. If it's a symbolic link, a portable way to resolve it is:

% file -h /bin/sh
/bin/sh: symbolic link to bash

If it's a hard link, try

% find -L /bin -samefile /bin/sh
/bin/sh
/bin/bash

In fact, the -L flag covers both symlinks and hardlinks, but the disadvantage of this method is that it is not portable — POSIX does not require find to support the -samefile option, although both GNU find and FreeBSD find support it.

Shebang line

Ultimately, it's up to you to decide which one to use, by writing the «shebang» line.

E.g.

#!/bin/sh

will use sh (and whatever that happens to point to),

#!/bin/bash

will use /bin/bash if it's available (and fail with an error message if it's not). Of course, you can also specify another implementation, e.g.

#!/bin/dash

Which one to use

For my own scripts, I prefer sh for the following reasons:

  • it is standardized
  • it is much simpler and easier to learn
  • it is portable across POSIX systems — even if they happen not to have bash, they are required to have sh

There are advantages to using bash as well. Its features make programming more convenient and similar to programming in other modern programming languages. These include things like scoped local variables and arrays. Plain sh is a very minimalistic programming language.

  • Thanks for the detailed answer: I use Linux Mint which is Debian-based and /bin/sh is indeed a symlink to /bin/dash. – Max Williams Jul 19 '17 at 8:32
  • If you want to adhere to POSIX sh, then best leave out the shebang completely: If the first line of a file of shell commands starts with the characters "#!", the results are unspecified pubs.opengroup.org/onlinepubs/009695399/utilities/… – Daniel Jour Jul 19 '17 at 16:28
  • 4
    @DanielJour If any Unix-like system dropped the usual support for shebangs it would break so many things it would be practically unusable. So it's a de facto standard, even if it's not specified in POSIX. The only actual compatibility issue is that the path to interpreters may differ. – Barmar Jul 19 '17 at 18:28
23

Adding to the excellent answer from @Hunter.S.Thompson I'd like to point out that the non-portable part of the script is

pdf_file="${html_file/.html/.pdf}"

The ${variable/search/replace} is a GNU extension. But you can easily avoid it with pure POSIX:

pdf_file="${html_file%.html}".pdf

Following Hunter, this is the better fix than changing the shebang to #! /bin/bash

  • Thanks - I agree that it's the "purer" fix but I'd rather the shebang loaded bash, as bash is what I routinely use in the terminal (by default) and it's easier to have the scripts just do the same thing as the regular command line. – Max Williams Jul 19 '17 at 10:36
  • 1
    @MaxWilliams Of course, no problem. This was mainly indended for the Q&A database. – Philippos Jul 19 '17 at 13:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.