17

There are variables in the shell like $0, $1, $2, $?, etc.

I tried to print the shell and environment variables using the following command:

set

But these variables were not in the list.

So basically these variables are not considered to be shell/environment variables, right? (even though to output them, you have to precede them with a $, like you do with shell/environment variables)

  • 3
    The positional parameters are not variables. You cannot export 3 to turn $3 into an environment variable. You cannot unset 3; and you cannot assign $3 a new value using 3=val. – Kaz Oct 26 '17 at 7:49
25

Variables are one of three distinct varieties of parameters in shell.

  1. A variable is a parameter whose name is a valid shell identifier; starts with _ or a letter, followed by zero or more letters, numbers or _.
  2. The positional parameters are the numbered parameters $1, $2, ...
  3. The special parameters all have single-character names, and aside from $0, they are all various punctuation characters.

set only displays the shell's variables.

A subset of the shell variables are the environment variables, whose values are either inherited from the environment when the shell starts up, or are created by setting the export attribute on a valid name.

  • 1
    Note that set displays all parameters in zsh (not $1, $2... but $*, $@) and the functions in bash and bosh. Some shells like ksh93 and older versions of dash output env vars that were not mapped to shell variables. (env 1=foo ksh -c set would print 1=foo) – Stéphane Chazelas Oct 26 '17 at 14:46
11

Environment Variables vs Positional Parameters

Before we begin discussing $INTEGER type of variables, we need to understand what they really are and how they differ from environment variables.Variables such as $INTEGER are called positional parameters. This is described in the POSIX (Portable Operating System Interface) standard, section 2.1 (emphasis mine):

  1. The shell executes a function (see Function Definition Command), built-in (see Special Built-In Utilities), executable file, or script, giving the names of the arguments as positional parameters numbered 1 to n, and the name of the command (or in the case of a function within a script, the name of the script) as the positional parameter numbered 0 (see Command Search and Execution).

By contrast, variables such as $HOME and $PATH are environment variables. Their definition is described in section 8 of the standard:

Environment variables defined in this chapter affect the operation of multiple utilities, functions, and applications. There are other environment variables that are of interest only to specific utilities. Environment variables that apply to a single utility only are defined as part of the utility description.

Notice their description. Positional parameters are meant to appear in front of a command, i.e. command positional_arg_1 positional_arg_2.... They are meant to be provided by user to tell command what specifically to do. When you do echo 'Hello' 'World', it will print out the Hello and World strings, because these are positional parameters to echo - the things you want echo to operate on. And echo is built such that it understands positional parameters as strings to be printed (unless they're one of the optional flags like -n ). If you do this with different command it might not understand what Hello and World is because perhaps it expects a number. Notice that positional parameters are not "inherited" - a child process does not know about positional parameters of the parent unless explicitly passed to the child process. Often you see positional parameters being passed with wrapper scripts - the ones that maybe check for already existing instance of a command or add additional positional parameters to the real command that will be called.

By contrast, environment variables are meant to affect multiple programs. They are environment variables, because they're set outside of the program itself (more on this below). Certain environment variables such as HOME or PATH have specific format, specific meaning, and they'll mean the same to each program. HOME variable will mean same to either external utility like /usr/bin/find or your shell ( and consequently to a script ) - it's the home directory of the username under which process runs. Notice that environmental variables can be used to account for specific command behavior, for instance UID environment variable can be used to check whether the script runs with root privileges or not and branch to specific actions accordingly. Environment variables are inheritable - child processes get copy of parent's environment. See also If processes inherit the parent's environment, why do we need export?

In short, the main distinction is that environment variables are set outside of the command and not meant to be varied (usually), while positional parameters are things that are meant to be processed by the command and they change.


Not just shell concepts

What I've noticed from comments is that you're mixing up terminal and shell, and would really recommend you read about real terminals that once upon a time were physical devices. Nowadays, the "terminal" that we're typically referring to, that window with black background and green text is actually software, a process. Terminal is a program that runs a shell, while shell is also a program but the one that reads what you type in to execute ( that is, if it is interactive shell; non-interactive shells are scripts and sh -c 'echo foo' types of invocations). More on shells here.

This is an important distinction, but also important to recognize that terminal is a program and therefore adheres to the same rules of environment and positional parameters. Your gnome-terminal when started will look at your SHELL environment variable, and spawn the appropriate default shell for you, unless you specify some other command with -e. Let's say I changed my default shell to ksh - gnome-terminal will then spawn ksh instead of bash. That's also an example of how environment is used by programs. If I explicitly tell gnome-terminal with -e to run specific shell - it will do it, but it won't be permanent. By contrast, environment is meant to be mostly unaltered (more on that later).

So as you can see, environment and positional variables are both properties of a process/command, not just shell. When it comes to shell scripts, they also follow the model that was set by C programming language. Take for instance the C main function which typically looks like

int main(int argc, char **argv)

, where argc is number of command-line arguments and argv is effectively array of command-line parameters, and then there's environ function ( on Linux that's man -e 7 environ ) to access things like user's home directory path, list of directories in PATH where we can look for executables, etc. Shell scripts are also modeled in the similar way. In shell terminology, we have positional parameters $1, $2 and so forth, while $# is number of positional parameters. What about $0 ? That's the name of the executable itself, which is again also modeled from C programming language - argv[0] would be name of your C "executable". And this is fairly true for most programming and scripting languages.

Interactive vs Non-interactive shells

One of the things I've already hinted is the distinction between interactive and non-interactive shells. The prompt where you type in commands - that's interactive, it interacts with the user. By contrast when you have a shell script or you run bash -c'' that's non-interactive.

And this is where distinction becomes important. The shell that you run already is a process, which was spawned with positional parameters ( for bash login shell is one "...whose first character of argument zero is a -, or one started with the --login option." (reference ) )

By contrast, scripts and shells launched with -c option can take advantage of $1 and $2 arguments. For instance,

$ bash -c 'echo $1; stat $2' sh 'Hello World' /etc/passwd
Hello World
  File: '/etc/passwd'
  Size: 2913        Blocks: 8          IO Block: 4096   regular file
Device: 801h/2049d  Inode: 6035604     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2017-08-12 14:48:37.125879962 -0600
Modify: 2017-08-12 14:48:37.125879962 -0600
Change: 2017-08-12 14:48:37.137879811 -0600
 Birth: -

Notice that I've also used sh there, because small quirk of -c option is to take first positional parameter and assign it to $0, unlike typically being a name of the program.

Another thing that is important to notice is that positional parameters are what I call "framable". Notice how, we first launched bash with its own positional parameters, but those positional parameters became parameters to echo and stat. And each program understands it in its own way. If we gave to stat a string Hello World and there's no file Hello World it would produce an error; bash treats it just as a simple string but stat expects that string to be an existing filename. By contrast, all programs would agree that environment variable HOME is a directory ( unless the programmer coded it in an unreasonable way).


Can we mess around with environment variables and positional parameters ?

Technically, we can mess around with both, but we shouldn't mess around with environment variables, while we often have to provide positional parameters. We can run commands in shell with prepending a variable, for instance:

$ hello=world bash -c 'echo $hello'
world

We can also place variables into environment simply using export variable=value from within shell or script. Or we can run a command with completely empty environment with env -c command arg1 arg2. However, typically it is not recommended to mess around with environment, especially using upper-case variables or overwriting already existing environment variables. Notice that's recommended although not a standard.

For positional parameters, the way to set them is obvious, just prepend them to the command, but also there are ways to set them other wise, as well as change the list of those parameters via shift command.

In conclusion, purpose of these two is different, and they exist for a reason. I hope people gained some insight from this answer, and it was fun reading it just as it was for me to write this answer.


Note on set command

The set command, according to manual behaves like so (from bash manual, emphasis added):

Without options, the name and value of each shell variable are displayed in a format that can be reused as input for setting or resetting the currently-set variables.

In other words set looks at variables specific to shell, some of which happen to be in the environment, for example HOME. By contrast commands like env and printenv look at actual environment variable with which a command runs. See also this.

  • " In interactive shell, you cannot reference $1, $2, and so forth. " is there a direct reference for this? I have run into strange cases where these are set in an interactive shell environment and I am not sure if this would be considered 'non-standard' or not. – user5359531 Dec 12 '18 at 17:35
  • @user5359531 Honestly, I'm not quite sure where I got that from. Probably because back when I posted the answer in 2017 I probably referenced that you can't do something like 1="foo" but later I found out that by POSIX definition a "word" ( that is a name of an object such as variable or function ) cannot start with a digit ( see a question I posted on the topic ). Positional parameters apparently are exception to this rule. – Sergiy Kolodyazhnyy Dec 13 '18 at 3:52
  • @user5359531 I've removed the part of the answer, since it's not particularly accurate. You can reference $1, $2 and so forth in interactive shell, and in fact that's done with set command, often to get around /bin/sh limitation of not having arrays. Thanks for bringing this up to my attention. I'm also going to edit the answer over the next few days, since it needs a bit of extra polish and an update. – Sergiy Kolodyazhnyy Dec 13 '18 at 3:54
  • Thanks for the clarification. The problem I am having is that with conda, when you run source conda/bin/activate, it checks for whether $1, $2, etc., are set, in order to determine if it was run as a script with arguments or not. This ends up breaking on systems that have those set in the interactive environment for some reason. I am hoping to figure out if this non-standard behavior is a flaw in the system for setting these variables in the interactive environment, or on the program for using them to determine if it was run as a script. – user5359531 Dec 13 '18 at 17:19
  • @user5359531 I'd suggest you file a bug report to conda developers or to whoever is the original author of such script, as checking for ${N} parameters is definitely the wrong way to go. There are questions on the same topic here and here, and more or less portable way is to check if ${0} is same as script name, while bash actually has environment variable for that purpose – Sergiy Kolodyazhnyy Dec 13 '18 at 18:51
4

The $1, $2, $3, ..., ${10}, ${11} variables are called positional parameters and are covered in the bash manual section 3.4.1

3.4.1 Positional Parameters

A positional parameter is a parameter denoted by one or more digits, other than the single digit 0. Positional parameters are assigned from the shell’s arguments when it is invoked, and may be reassigned using the set builtin command. Positional parameter N may be referenced as ${N}, or as $N when N consists of a single digit. Positional parameters may not be assigned to with assignment statements. The set and shift builtins are used to set and unset them (see Shell Builtin Commands). The positional parameters are temporarily replaced when a shell function is executed (see Shell Functions).

When a positional parameter consisting of more than a single digit is expanded, it must be enclosed in braces.

As for $? and $0, these special parameters are covered in the very next section 3.4.2

3.4.2 Special Parameters

The shell treats several parameters specially. These parameters may only be referenced; assignment to them is not allowed.

...

?

($?) Expands to the exit status of the most recently executed foreground pipeline.

0

($0) Expands to the name of the shell or shell script. This is set at shell initialization. If Bash is invoked with a file of commands (see Shell Scripts), $0 is set to the name of that file. If Bash is started with the -c option (see Invoking Bash), then $0 is set to the first argument after the string to be executed, if one is present. Otherwise, it is set to the filename used to invoke Bash, as given by argument zero.

4

$1, $2... are positional parameters, they are not variables, let alone environment variables.

In Bourne-like shell terminology, $something is called parameter expansion (also covers ${something#pattern} and more in some shells like ${array[x]}, ${param:offset}, ${x:|y} and many more expansion operators).

There are different sorts of parameters:

  • variables like $foo, $PATH
  • positional parameters ($1, $2... the arguments that your script received)
  • other special parameters like $0, $-, $#, $*, $@, $$, $!, $?...

variable names in Bourne like shells must start with one alphabetical character (any recognised by the locale, or limited to a-zA-Z depending on the shell) and underscore and followed by zero or more alphanumerical characters or underscores.

Depending on the shell, variables can have different types (scalar, array, hash) or given certain attributes (read-only, exported, lower case...).

Some of those variables are created by the shell or have special meaning to the shell (like $OPTIND, $IFS, $_...)

Shell variables that have the export attribute are automatically exported as environment variables to the commands that the shell executes.

Environment variable is a concept separate from shell variables. Exporting a shell variable is not the only way to pass an environment variable to the execution of a command.

VAR=foo
export VAR
printenv VAR

will pass a VAR environment variable to the printenv command (which we're telling to print its content), but you can also do:

env VAR=foo printenv VAR

or:

perl -e '$ENV{VAR}="foo"; exec "printenv", "VAR"'

for instance.

Environment variables can have any name (can contain any character but = and can even be empty). It's not a good idea to give a name that is not compatible to that of a Bourne-like shell variable name to an environment variable, but it's possible:

$ env '#+%=whatever' printenv '#+%'
whatever

Shells will map the environment variables that they receive to shell variables only for those environment variables whose name are valid shell variables (and in some shells ignore some special ones like $IFS).

So, while you'd be able to pass a 1 environment variable to a command:

$ env '1=whatever' printenv 1
whatever

That doesn't mean that calling a shell with that environment variable would set the value of the $1 parameter:

$ env '1=whatever' sh -c 'echo "$1"' script-name foo bar
foo
3

No, these are parameters of the script. For example if you call your script like:

mynicescript.sh one two three

then inside the script, there will be these parameters available as

$1 = one
$2 = two
$3 = three

and the $0 is the name of the script itself.

So when you're outside the script, these variables aren't available (except $0, which displays /bin/bash - the shell itself).

  • "So when you're outside the script, these variables aren't available" What do you mean by "outside the script", because I can see the values of these variables in my terminal. – user7681202 Oct 25 '17 at 22:30
  • 2
    @user7681202: Which ones can you see in your terminal? $0 will point to your current terminal process (likely bash) and $? is simply the exit code of the last process. – Jesse_b Oct 25 '17 at 22:33
  • I tried to run the gnome-terminal with arguments (gnome-terminal Hello World). I could see $0, but I could not see $1 and $2. – user7681202 Oct 25 '17 at 22:36
  • @Jesse_b Thank you, I've added the content of $0 into the answer. – Jaroslav Kucera Oct 25 '17 at 22:37
  • 1
    @user7681202 The gnome-terminal isn't shell, it's terminal emulator (like xterm, konsole etc.). The shell runs inside the terminal and it can be bash/sh/zsh/tcsh and many more. The script is executable file with proper header (like #!/bin/bash) and content interpretable by the shell specified in the mask. It usually uses suffix .sh – Jaroslav Kucera Oct 25 '17 at 22:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.