Multiple arguments in shebang

Question

I am wondering whether there is a general way of passing multiple options to an executable via the shebang line (#!).

I use NixOS, and the first part of the shebang in any script I write is usually /usr/bin/env. The problem I encounter then is that everything that comes after is interpreted as a single file or directory by the system.

Suppose, for example, that I want to write a script to be executed by bash in posix mode. The naive way of writing the shebang would be:

#!/usr/bin/env bash --posix

but trying to execute the resulting script produces the following error:

/usr/bin/env: ‘bash --posix’: No such file or directory

I am aware of this post, but I was wondering whether there was a more general and cleaner solution.

---

EDIT: I know that for Guile scripts, there is a way to achieve what I want, documented in Section 4.3.4 of the manual:

 #!/usr/bin/env sh
 exec guile -l fact -e '(@ (fac) main)' -s "$0" "$@"
 !#

The trick, here, is that the second line (starting with exec) is interpreted as code by sh but, being in the #! ... !# block, as a comment, and thus ignored, by the Guile interpreter.

Would it not be possible to generalize this method to any interpreter?

---

Second EDIT: After playing around a little bit, it seems that, for interpreters that can read their input from stdin, the following method would work:

#!/usr/bin/env sh
sed '1,2d' "$0" | bash --verbose --posix /dev/stdin; exit;

It's probably not optimal, though, as the sh process lives until the interpreter has finished its job. Any feedback or suggestion would be appreciated.

Answer 1

Although not exactly portable, starting with coreutils 8.30 and according to its documentation you will be able to use:

#!/usr/bin/env -S command arg1 arg2 ...

So given:

$ cat test.sh
#!/usr/bin/env -S showargs here 'is another' long arg -e "this and that " too

you will get:

% ./test.sh 
$0 is '/usr/local/bin/showargs'
$1 is 'here'
$2 is 'is another'
$3 is 'long'
$4 is 'arg'
$5 is '-e'
$6 is 'this and that '
$7 is 'too'
$8 is './test.sh'

and in case you are curious showargs is:

#!/usr/bin/env sh
echo "\$0 is '$0'"

i=1
for arg in "$@"; do
    echo "\$$i is '$arg'"
    i=$((i+1))
done

Answer 2

There is no general solution, at least not if you need to support Linux, because the Linux kernel treats everything following the first “word” in the shebang line as a single argument.

I’m not sure what NixOS’s constraints are, but typically I would just write your shebang as

#!/bin/bash --posix

or, where possible, set options in the script:

set -o posix

Alternatively, you can have the script restart itself with the appropriate shell invocation:

#!/bin/sh -

if [ "$1" != "--really" ]; then exec bash --posix -- "$0" --really "$@"; fi

shift

# Processing continues

This approach can be generalised to other languages, as long as you find a way for the first couple of lines (which are interpreted by the shell) to be ignored by the target language.

GNU coreutils’ env provides a workaround since version 8.30, see unode’s answer for details. (This is available in Debian 10 and later, RHEL 8 and later, Ubuntu 19.04 and later, etc.)

Answer 3

The POSIX standard is very terse on describing #!:

From the rationale section of the documentation of the exec() family of system interfaces:

Another way that some historical implementations handle shell scripts is by recognizing the first two bytes of the file as the character string #! and using the remainder of the first line of the file as the name of the command interpreter to execute.

From the Shell Introduction section:

The shell reads its input from a file (see sh), from the -c option or from the system() and popen() functions defined in the System Interfaces volume of POSIX.1-2008. If the first line of a file of shell commands starts with the characters #!, the results are unspecified.

This basically means that any implementation (the Unix you are using) is free to do the specifics of the parsing of the shebang line as it wants.

Some Unices, like macOS (can't test ATM), will split the arguments given to the interpreter on the shebang line into separate arguments, while Linux and most other Unices will give the arguments as a single option to the interpreter.

It is thus unwise to rely on the shebang line being able to take more than a single argument.

See also the Portability section of the Shebang article on Wikipedia.

---

One easy solution, which is generalizable to any utility or language, is to make a wrapper script that executes the real script with the appropriate command line arguments:

#!/bin/sh
exec /bin/bash --posix /some/path/realscript "$@"

I don't think I would personally try to make it re-execute itself as that feels somewhat fragile.

Answer 4

I found a rather stupid workaround when looking for an executable that accepts a script as a single argument:

#!/usr/bin/awk BEGIN{system("bash --posix "ARGV[1])}

Answer 5

The shebang is described in execve(2) man page as follow:

#! interpreter [optional-arg]

Two spaces are accepted in this syntax:

1. One space before the interpreter path, but this space is optional.
2. One space separating the the interpreter path and its optional argument.

Note that I didn't used the plural when talking of an optional argument, neither does the syntax above uses [optional-arg ...], as you can provide at most one single argument.

As far as shell scripting is concerned, you can use the set built-in command near the beginning of your script which will allow to set interpreters parameters, providing the same result as if you used command-line arguments.

In your case:

set -o posix

From a Bash prompt, check the output of help set to get all available options.

Answer 6

On Linux, the shebang isn't very flexible; according to multiple answers (Stephen Kitt's answer and Jörg W Mittag's), there is no designated way to pass multiple arguments in a shebang line.

I'm not sure if it will be of use to anyone, but I've written a short script to implement the lacking feature. See https://gist.github.com/loxaxs/7cbe84aed1c38cf18f70d8427bed1efa.

It is also possible to write embedded workarounds. Bellow, I present four language-agnostic workarounds applied to the same test script and the result each prints. I suppose that the script is executable and is in /tmp/shebang.

---

Wrapping your script in a bash heredoc inside process substitution

As far as I know, this is the most reliable language-agnostic way of doing it. It allows passing arguments and preserves stdin. The drawback is that the interpreter doesn't know the (real) location of the file it reads.

#!/bin/bash
exec python3 -O <(cat << 'EOWRAPPER'
print("PYTHON_SCRIPT_BEGINNING")

from sys import argv
try:
    print("input() 0 ::", input())
    print("input() 1 ::", input())
except EOFError:
    print("input() caused EOFError")
print("argv[0]   ::", argv[0])
print("argv[1:]  ::", argv[1:])
print("__debug__ ::", __debug__)
# The -O option changes __debug__ to False

print("PYTHON_SCRIPT_END")
EOWRAPPER
) "$@"

Calling echo -e 'aa\nbb' | /tmp/shebang 'arg1' 'arg2 contains spaces' 'arg3\ uses\ \\escapes\\' prints:

PYTHON_SCRIPT_BEGINNING
input() 0 :: aa
input() 1 :: bb
argv[0]   :: /dev/fd/62
argv[1:]  :: ['arg1', 'arg2 contains spaces', 'arg3\\ uses\\ \\\\escapes\\\\']
__debug__ :: False
PYTHON_SCRIPT_END

Note that process substitution produces a special file. This may not suit all executables. For instance, #!/usr/bin/less complains: /dev/fd/63 is not a regular file (use -f to see it)

I don't know if it is possible to have heredoc inside process substitution in dash.

---

Wrapping your script in a simple heredoc

Shorter and simpler, but you won't be able to access stdin from your script and it requires the interpreter to be able to read and execute a script from stdin.

#!/bin/sh
exec python3 - "$@" << 'EOWRAPPER'
print("PYTHON_SCRIPT_BEGINNING")

from sys import argv

try:
    print("input() 0 ::", input())
    print("input() 1 ::", input())
except EOFError:
    print("input() caused EOFError")
print("argv[0]   ::", argv[0])
print("argv[1:]  ::", argv[1:])
print("__debug__ ::", __debug__)
# The -O option changes __debug__ to False

print("PYTHON_SCRIPT_END")
EOWRAPPER

Calling echo -e 'aa\nbb' | /tmp/shebang 'arg1' 'arg2 contains spaces' 'arg3\ uses\ \\escapes\\' prints:

PYTHON_SCRIPT_BEGINNING
input() caused EOFError
argv[0]   :: -
argv[1:]  :: ['arg1', 'arg2 contains spaces', 'arg3\\ uses\\ \\\\escapes\\\\']
__debug__ :: True
PYTHON_SCRIPT_END

---

Use awk system() call but without arguments

Correctly passes the name of the executed file, but your script won't receive the arguments you give it. Note that awk is the only language I know whose interpreter both is installed on linux by default and reads its instructions from the command line by default.

#!/usr/bin/gawk BEGIN {system("python3 -O " ARGV[1])}
print("PYTHON_SCRIPT_BEGINNING")

from sys import argv

print("input() 0 ::", input())
print("input() 1 ::", input())
print("argv[0]   ::", argv[0])
print("argv[1:]  ::", argv[1:])
print("__debug__ ::", __debug__)
# The -O option changes __debug__ to False

print("PYTHON_SCRIPT_END")

Calling echo -e 'aa\nbb' | /tmp/shebang 'arg1' 'arg2 contains spaces' 'arg3\ uses\ \\escapes\\' prints:

PYTHON_SCRIPT_BEGINNING
input() 0 :: aa
input() 1 :: bb
argv[0]   :: /tmp/shebang
argv[1:]  :: []
__debug__ :: False
PYTHON_SCRIPT_END

---

Use awk 4.1+ system() call, provided your arguments do not contain spaces

Nice, but only if you are sure your script won't be called with arguments containing spaces. As you can see, your arguments containing spaces would be split, unless the spaces are escaped.

#!/usr/bin/gawk @include "join"; BEGIN {system("python3 -O " join(ARGV, 1, ARGC, " "))}
print("PYTHON_SCRIPT_BEGINNING")

from sys import argv

print("input() 0 ::", input())
print("input() 1 ::", input())
print("argv[0]   ::", argv[0])
print("argv[1:]  ::", argv[1:])
print("__debug__ ::", __debug__)
# The -O option changes __debug__ to False

print("PYTHON_SCRIPT_END")

Calling echo -e 'aa\nbb' | /tmp/shebang 'arg1' 'arg2 contains spaces' 'arg3\ uses\ \\escapes\\' prints:

PYTHON_SCRIPT_BEGINNING
input() 0 :: aa
input() 1 :: bb
argv[0]   :: /tmp/shebang
argv[1:]  :: ['arg1', 'arg2', 'contains', 'spaces', 'arg3 uses \\escapes\\']
__debug__ :: False
PYTHON_SCRIPT_END

For awk versions below 4.1, you will have to use string concatenation inside a for loop, see example function https://www.gnu.org/software/gawk/manual/html_node/Join-Function.html .

Answer 7

A trick to use LD_LIBRARY_PATH with python on the #! (shebang) line that doesn't depend on anything else than the shell and works a treat:

#!/bin/sh
'''' 2>/dev/null; exec /usr/bin/env LD_LIBRARY_PATH=. python -x "$0" "$@" #'''

__doc__ = 'A great module docstring'

As explained elsewhere in this page some shells like sh can take a script on their standard input.

The script we give sh tries to execute the command '''' which is simplified to '' (the empty string) by sh and of course it fails to execute it as there are no '' command, so it normally outputs line 2: command not found on the standard error descriptor but we redirect this message using 2>/dev/null to the closest black hole because it'd be messy and confusing to the user to let sh display it.

We then proceed to the command of interest to us: exec which replaces the current shell process by what follows, in our case: /usr/bin/env python with the adequate parameters:

• "$0" to let python know which script it should open and interpret, and also set sys.argv[0]
• "$@" to set python's sys.argv[1:] to the arguments passed on the script command line.

And we also ask env to set the LD_LIBRARY_PATH environment variable, which is the sole point of the hack.

The shell command ends at the comment starting with # so that the shell ignores the trailing triple quotes '''.

sh is then replaced by a shinny new instance of the python interpreter which open and read the python source script given as first argument (the "$0").

Python opens the file and skips over the 1st line of the source thanks to the -x argument. Note: it also works without -x because for Python a shebang is just a comment.

Python then interprets the 2nd line as the docstring for the current module file, so if you need a valid module docstring, just set __doc__ first thing in your python program as in the example above.

Answer 8

Coming a little late to this question but I had the issue that I wanted multiple arguments to my command, and also to pass command line args from the shebang invocation to the interpreter.

I put together this awk script:

#!/usr/bin/awk BEGIN{ for (x = 1; x < ARGC; x++) {args=args sprintf("\"%s\" ", ARGV[x]); ARGV[i]=""} cmd = "COMMAND_HERE WITH_ARGS" args; system(cmd)}

Which allows me to execute the script as ./script 1 2, and my command will be executed with the hardcoded args followed by the args the script was invoked with. This will give the script the name it is invoked with in COMMAND_HERE as the first arg, and the name that you invoked the script with in the second (If you don't want this, change the starting index in the for to 2), and not split quoted arguments containing spaces (If you want to split them, remove the escaped quotes around %s in the sprintf() call). This will also work with redirection and piping. I believe this is portable as it only uses basic builtin Awk features.