12

I have a project comprised of about 20 small .sh files. I name these "small" because generally, no file has more than 20 lines of code. I took a modular approach because thus I'm loyal to the Unix philosophy and it's easier for me to maintain the project.

In the start of each .sh file, I put #!/bin/bash.

Simply put, I understand script declarations have two purposes:

  1. They help the user recall what shell is needed to execute the file (say, after some years without using the file).
  2. They ensure that the script runs only with a certain shell (Bash in that case) to prevent unexpected behavior in case another shell was used.

When a project starts to grow from say 5 files to 20 files or from 20 files to 50 files (not this case but just to demonstrate) we have 20 lines or 50 lines of script declarations. I admit, even though it might be funny to some, it feels a bit redundant for me to use 20 or 50 instead say just 1 per project (maybe in the main file of the project).

Is there a way to avoid this alleged redundancy of 20 or 50 or a much greater number of lines of script declarations by using some "global" script declaration, in some main file?

  • 2
    You should not but could; remove it and execute all your scripts with /bin/bash -x "$script" – Jesse_b Feb 3 '18 at 12:57
  • 2
  • ... Usually by the time something gets to this point I've already concluded it's time to stop using shell scripts and get an actual scripting language to do the work. – Shadur Feb 10 '18 at 11:49
19
+50

Even though your project may now be solely consisting of 50 Bash scripts, it will sooner or later start accumulating scripts written in other languages such as Perl or Python (for the benefits that these scripting languages have that Bash does not have).

Without a proper #!-line in each script, it would be extremely difficult to use the various scripts without also knowing what interpreter to use. It doesn't matter if every single script is executed from other scripts, this only moves the difficulty from the end users to the developers. Neither of these two groups of people should need to know what language a script was written in to be able to use it.

Shell scripts executed without a #!-line and without an explicit interpreter are executed in different ways depending on what shell invokes them (see e.g. the question Which shell interpreter runs a script with no shebang? and especially Stéphane's answer), which is not what you want in a production environment (you want consistent behaviour, and possibly even portability).

Scripts executed with an explicit interpreter will be run by that interpreter regardless of what the #!-line says. This will cause problems further down the line if you decide to re-implement, say, a Bash script in Python or any other language.

You should spend those extra keystrokes and always add a #!-line to each and every script.


In some environments, there are multi-paragraph boilerplate legal texts in each script in each project. Be very happy that it's only a #!-line that feels "redundant" in your project.

  • The answer should be expanded by this: The shell determines the interpreter by the #! line as soon as the file has exectuable permission and is not loaded by an interpreter, but by the shell. That is, /path/to/myprog.pl runs a perl program if the #! line is present (pointing to the perl interpreter) and the file has exec permission. – rexkogitans Feb 4 '18 at 14:52
11

You misunderstand the point of #!. It's actually a directive to the operating system to run this program under the defined interpreter.

So a script could being #!/usr/bin/perl and the resulting program can be run as ./myprogram and the perl interpreter could be used. Similar #!/usr/bin/python would cause a program to run under python.

So the line #!/bin/bash tells the OS to run this program under bash.

Here's a example:

$ echo $0
/bin/ksh

$ cat x
#!/bin/bash

ps -aux | grep $$

$ ./x
sweh      2148  0.0  0.0   9516  1112 pts/5    S+   07:58   0:00 /bin/bash ./x
sweh      2150  0.0  0.0   9048   668 pts/5    S+   07:58   0:00 grep 2148

So, despite my shell being ksh, the program "x" runs under bash because of the #! line, and we can see that explicitly in the process listing.

  • ps -aux may give warning, – Weijun Zhou Feb 3 '18 at 13:00
  • @WeijunZhou Say more... – Kusalananda Feb 3 '18 at 13:01
  • Some version of ps gives the warning Warning: bad syntax, perhaps a bogus '-'?. – Weijun Zhou Feb 3 '18 at 13:03
  • 2
    ps suooprts both BSD and UXIX argument syntax. -aux is wrong UNIX Stephen probably means aux which is correct BSD – Jasen Feb 3 '18 at 20:27
  • 1
    People, 2 things; (1) focus on the the example concept (that the ps shows the calling of bash) not the explicit syntax; (2) ps -aux works perfectly on CentOS 7 and Debian 9 without warnings; that cut'n'paste was exactly the output from my machine; the whole discussion of if the - is needed or not is applicable to older versions of linux procps, not current versions. – Stephen Harris Feb 4 '18 at 15:38
9

On .sh

What is bad is the .sh at the end of the file-name.

Imagine that you rewrite one of the scripts in python.

Well now you have to change the first line to #!/usr/bin/python3 this is not so bad, as you had to change every other line of code in the file as well. However you also have to change the file name from prog.sh to prog.py. And then you have to find everywhere in the other scripts, and all of your user scripts (the ones that you did not even know existed), and change them to use the new script. sed -e 's/.sh/.py/g' may help for the ones you know about. But now your revision control tool is showing a change in lots of unrelated files.

Alternatively do it the Unix way and name the program prog not prog.sh.

On #!

If you have the correct #!, and set the permission/mode to include execute. Then you don't need to know the interpreter. The computer will do it for you.

chmod +x prog #on gnu adds execute permission to prog.
./prog #runs the program.

For non gnu systems read the manual for chmod (and learn octal), maybe read it anyway.

  • 1
    Hmmm, extensions aren't necessarily bad, at least they help communicating with other users what's the file type. – Sergiy Kolodyazhnyy Feb 3 '18 at 17:01
  • @SergiyKolodyazhnyy But you don't need to know the language, you just need to run it. Even Microsoft is trying to move away from extensions (unfortunately the are doing this my hiding them). However I do agree that they could be a good idea: .image for pictures. .audio for sound etc. Thus telling you what it is but not about the implementation/encoding. – ctrl-alt-delor Feb 3 '18 at 17:06
  • 1
    @ctrl-alt-delor If the file is called launch-missiles or delete-project-and-make-manager-pissed I don't wanna "just run it" when I was only curious about the language :) – Sergiy Kolodyazhnyy Feb 3 '18 at 17:08
  • 2
    if you're curious for the language use the file command. it recognises most file types. – Jasen Feb 3 '18 at 20:38
  • 2
    I agree that extensions are bad for executable scripts, for exactly the reasons given.I use a .sh extension for a script that needs to be sourced by another shell script (i.e. a non-executable script) - in that scenario, the suffix is important/valid because it tells us how we can use the file. – Laurence Renshaw Feb 9 '18 at 0:54
8

[The hashbang lines] ensure that the script runs only with a certain shell (Bash in that case) to prevent unexpected behavior in case another shell was used.

It's probably worth pointing out that this is quite wrong. The hashbang line in no way prevents you from trying to feed the file to an incorrect shell/interpreter:

$ cat array.sh
#!/bin/bash
a=(a b c)
echo ${a[1]} $BASH_VERSION
$ dash array.sh
array.sh: 2: array.sh: Syntax error: "(" unexpected

(or similarly with awk -f array.sh etc.)


But in any case, another approach to modularity would be to define shell functions in the files, one function per file, if you like, and then source the files from the main program. That way, you wouldn't need hashbangs: they'd be treated as any other comments, and everything would run using the same shell. The downside would be that you'd need to source the files with the functions (e.g. for x in functions/*.src ; do source "$x" ; done), and they'd all run using the same shell, so you couldn't directly replace one of them with a Perl-implementation.

  • +1 for example with bash arrays. Users unfamiliar with Multiple shells or some of what POSIX defines, may assume some features of one shell exist in another. Also for functions, this may be relevant: unix.stackexchange.com/q/313256/85039 – Sergiy Kolodyazhnyy Feb 3 '18 at 16:46
4

Is there a way to avoid this alleged redundancy of 20 or 50 or a much greater number of lines of script declarations by using some "global" script declaration, in some main file?

There is - it is called portability or POSIX compliance. Strive to always write scripts in portable, shell-neutral way, source them only from a script that uses #!/bin/sh or when you use /bin/sh interactively (or at very least a POSIX-compliant shell such as ksh).

However, portable scripts don't save you from:

  • needing to use features of one of the shells ( ilkkachu's answer is an example)
  • needing to use scripting languages more advanced than shells (Python and Perl)
  • PEBKAC error: your script falls into hands of user J.Doe who runs your script not as you intended it, for example they run /bin/sh script with csh.

If I may express my opinion, "avoiding redundancy" by eliminating #! is wrong goal. The goal should be consistent performance and actually a working script that works, and considers corner cases, follows certain general rules like not-parsing-ls or always-quoting-variables-unless-need-word-splitting.

They help the user recall what shell is needed to execute the file (say, after some years without using the file).

They're really not for user - they're for operating system to run proper interpreter. "Proper" in this case, means OS finds what is in shebang; if the code below it for wrong shell - that's author's fault. As for user memory, I don't think "user" of programs such as Microsoft Word or Google Chrome ever needed to know what language programs are written in; authors may need.

Then again, scripts written portably may eliminate need to even remember which shell you used originally, because portable scripts should work in any POSIX-compliant shell.

They ensure that the script runs only with a certain shell (Bash in that case) to prevent unexpected behavior in case another shell was used.

They don't. What actually prevents unexpected behavior is writing portable scripts.

1

The reason why you need to put #!/bin/bash at the top of each script is that you're running it as a separate process.

When you run a script as a new process, the OS sees that it's a text file (as opposed to a binary executable), and needs to know what interpreter to execute. If you don't tell it, the OS can only guess, using its default setting (which an administrator can change). So the #! line (plus adding executable permissions, of course) turns a simple text file into an executable that can be run directly, with confidence that the OS knows what to do with it.

If you want to remove the #! line, then your options are:

  1. Execute the script directly (as you're doing now), and hope that the default interpreter matches the script - you're probably safe on most Linux systems, but not portable to other systems (e.g. Solaris), and it can be changed to anything (I could even set it to run vim on the file!).

  2. Execute the script using bash myscript.sh. This works fine, but you have to specify the path to the script, and it's messy.

  3. Source the script using . myscript.sh, which runs the script within the current interpreter process (i.e. not as a sub-process). But this will almost certainly require modifications to your scripts, and makes them less modular / reusable.

In cases 2 and 3, the file doesn't need (and shouldn't have) execute permissions, and giving it a .sh (or .bash) suffix is a good idea.

But none of those options looks good for your project, as you said you want to keep your scripts modular (=> independent and self-contained), so you should keep your #!/bin/bash line at the top of the scripts.

In your question, you also said you're following the Unix philosophy. If that's true, then you should learn what the #! line is really for, and keep using it in every executable script!

  • Thanks for the good answer. I loved everything in the answer and thought of upvoting until this: then you should learn what the #! line is really for, and keep using it in every executable script!. One can follow the U.Philosophy up o certain point of knowledge. After all these answers I understood why it can be included in that term. I suggest editing the answer, replacing it with "See the shebang as an internal part of the Unix Philosophy which you respect and adopt". – user9303970 Feb 9 '18 at 2:45
  • 1
    Sorry if I was direct, or if that comment seemed rude. Your comments suggested that you prefer to work in an IDE - with a 'project' that allows you to define global rules for the scripts within that project. That's a valid way to develop, but it doesn't fit well with treating each script as an independent, standalone program (the Unix philosophy' you referred to). – Laurence Renshaw Feb 10 '18 at 3:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.