15

I actually did not know there are two different types of variables I can access from the command line. All I knew is, that I can declare variables like:

foo="my dear friends"
bar[0]="one"
bar[1]="two"
bar[2]="three"

or accessing them with a $ sign, like:

echo $foo
echo ${bar[1]}

or using inbuilt variables, like:

echo $PWD
PATH=$PATH:"/usr/bin/myProg"

Now, I hear there are two (at least?) types of variables: shell variables and environment variables.

  • What is the purpose of having two different types?
  • How do I know which type a variable is?
  • What are the typical usages for each one?
14

Environment variables are a list of name=value pairs that exist whatever the program is (shell, application, daemon…). They are typically inherited by children processes (created by a fork/exec sequence): children processes get their own copy of the parent variables.

Shell variables do exist only in the context of a shell. They are only inherited in subshells (i.e. when the shell is forked without an exec operation). Depending on the shell features, variables might not only be simple strings like environment ones but also arrays, compound, typed variables like integer or floating point, etc.

When a shell starts, all the environment variables it inherits from its parent become also shell variables (unless they are invalid as shell variables and other corner cases like IFS which is reset by some shells) but these inherited variables are tagged as exported1. That means they will stay available for children processes with the potentially updated value set by the shell. That is also the case with variables created under the shell and tagged as exported with the export keyword.

Array and other complex type variables cannot be exported unless their name and value can be converted to the name=value pattern, or when a shell specific mechanism is in place (e.g.: bash exports functions in the environment and some exotic, non POSIX shells like rc and es can export arrays).

So the main difference between environment variables and shell variables is their scope: environment variables are global while non exported shell variables are local to the script.

Note also that modern shells (at least ksh and bash) support a third shell variables scope. Variables created in functions with the typeset keyword are local to that function (The way the function is declared enables/disables this feature under ksh, and persistence behavior is different between bash and ksh). See https://unix.stackexchange.com/a/28349/2594

1This applies to modern shells like ksh, dash, bash and similar. The legacy Bourne shell and non Bourne syntax shells like csh have different behaviors.

  • 1
    Everything is inherited by children processes as children are created as a fork (an exact copy) of their parent. The point with environment variables is that they are passed to the execve() system call so are (typically) used to persist data over the execution of other commands (in the same process). – Stéphane Chazelas May 8 '17 at 8:51
  • Not all environment variables are translated to shell variables. Only those that are valid as a shell variable name (and with a few exceptions like IFS in some shells). – Stéphane Chazelas May 8 '17 at 8:53
  • Shells like rc, es can export arrays using a adhoc encoding. bash and rc can also export functions using environment variables (again, using a special encoding). – Stéphane Chazelas May 8 '17 at 8:54
  • In ksh93, typeset restricts the scope only in functions declared with the function foo { ...; } syntax, not with the Bourne (foo() cmd) syntax (and it's static scoping not dynamic like in other shells). – Stéphane Chazelas May 8 '17 at 8:57
  • @StéphaneChazelas Thanks for reviewing! Reply updated to take into account your remarks. – jlliagre May 8 '17 at 9:08
17

Shell variables

Shell variables are variables whose scope is in the current shell session, for example in an interactive shell session or a script.

You may create a shell variable by assigning a value to an unused name:

var="hello"

The use of shell variables is to keep track of data in the current session. Shell variables usually have names with lower-case letters.

Environment variables

An environment variable is a shell variable which has been exported. This means that it will be visible as a variable, not only in the shell session that created it, but also for any process (not just shells) that are started from that session.

VAR="hello"  # shell variable created
export VAR   # variable now part of the environment

or

export VAR="hello"

Once a shell variable has been exported, it stays exported until it is unset, or until its "export property" is removed (with export -n in bash), so there's usually no need to re-export it. Unsetting a variable with unset deletes it (no matter if it's an environment variable or not).

Arrays and associative hashes in bash and other shells may not be exported to become environment variables. Environment variables must be simple variables whose values are strings, and they often have names consisting of upper-case letters.

The use of environment variables is to keep track of data in the current shell session, but also to allow any started process to take part of that data. The typical case of this is the PATH environment variable, which may be set in the shell and later used by any program that wants to start programs without specifying a full path to them.

The collection of environment variables in a process is often referred to as "the environment of the process". Each process has its own environment.

Environment variables can only be "forwarded", i.e. a child process can never change the environment variables in its parent process, and other than setting up the environment for a child process upon starting it, a parent process may not change the existing environment of a child process.

Environment variables may be listed with env (without any arguments). Other than that, they appear the same as non-exported shell variables in a shell session. This is a bit special for the shell as most other programming languages don't usually intermix "ordinary" variables with environment variables (see below).

env may also be used to set the values of one or several environment variables in the environment of a process without setting them in the current session:

env CC=clang CXX=clang++ make

This starts make with the environment variable CC set to the value clang and CXX set to clang++.

It may also be used to clear the environment for a process:

env -i bash

This starts bash but does not transfer the current environment to the new bash process (it will still have environment variables as it creates new ones from its shell initialization scripts).

Example of difference

$ var="hello"   # create shell variable "var"
$ bash          # start _new_ bash session
$ echo "$var"   # no output
$ exit          # back to original shell session
$ echo "$var"   # "hello" is outputted
$ unset var     # remove variable

$ export VAR="hello"  # create environment variable "VAR"
$ bash
$ echo "$VAR"         # "hello" is outputted since it's exported
$ exit                # back to original shell session
$ unset VAR           # remove variable

$ ( export VAR="hello"; echo "$VAR" )  # set env. var "VAR" to "hello" in subshell and echo it
$ echo "$VAR"         # no output since a subshell has its own environment

Other languages

There are library functions in most programming languages that allows for getting and setting the environment variables. Note that since environment variables are stored as a simple key-value relationship, they are not usually "variables" of the language. A program may fetch the value (which is always a character string) corresponding to a key (the name of the environment variable), but will then have to convert it to an integer or whatever data type the language expects the value to have.

In C, environment variables may be accessed using getenv(), setenv(), putenv() and unsetenv(). Variables created with these routines are inherited in the same way by any process that the C program starts.

Other languages may have special data structures for accomplishing the same thing, like the %ENV hash in Perl, or the ENVIRON associative array in most implementations of awk.

  • thx, brilliantly clear explanation. So the environment is like a big field wherein other programs can live and see each of the environment variables. Some programs have their private variables, only they themselves can see them, like the shell. but there is a mechanism to make private variables being seen by all called "export". If this is ok understood, than the only thing i am not sure about is if there can exist more than one environment at same time? – sharkant May 7 '17 at 13:42
  • @sharkant Each running process has their own environment. This environment is inherited from the process that started it. There is never "cross-talk" between environments of different processes. The only way to change an environment variable in a process is for the process itself modifying it. – Kusalananda May 7 '17 at 13:44
  • thx for claryfing my understanding. Each fish inside its own fish bowl. How about Processes which spawn other processes? Are processes and their child-processes all inside one environment or has each its own? – sharkant May 7 '17 at 13:52
  • 1
    @sharkant There are library functions in most languages that allows for getting and setting the environment variables. In C, this is done with getenv(), setenv(), putenv() and unsetenv(). Variables created with these routines are inherited in the same way by any process that the C program starts. Other languages may have special data structures for the same thing, like %ENV in Perl. – Kusalananda May 7 '17 at 14:09
  • 1
    FWIW: the exec*() family of functions can also set the environment for the process being executed. – Satō Katsura May 8 '17 at 6:54
5

Shell variables are difficult to duplicate.

$ FOO=bar
$ FOO=zot
$ echo $FOO
zot
$ 

Environment variables however can be duplicated; they are just a list, and a list can have duplicate entries. Here's envdup.c to do just that.

#include <err.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

extern char **environ;

int main(int argc, char *argv[]) {
    char **newenv;
    int envcount = 0;

    if (argc < 2) errx(64, "Usage: envdup command [args ..]");

    newenv = environ;
    while (*newenv++ != NULL) envcount++;

    newenv = malloc(sizeof(char *) * (envcount + 3));
    if (newenv == NULL) err(1, "malloc failed");
    memcpy(newenv, environ, sizeof(char *) * envcount);
    newenv[envcount]   = "FOO=bar";
    newenv[envcount+1] = "FOO=zot";
    newenv[envcount+2] = NULL;

    environ = newenv;
    argv++;
    execvp(*argv, argv);
    err(1, "exec failed '%s'", *argv);
}

Which we can compile and run telling envdup to then run env to show us what environment variables are set...

$ make envdup
cc     envdup.c   -o envdup
$ unset FOO
$ ./envdup env | grep FOO
FOO=bar
FOO=zot
$ 

This is perhaps only useful for finding bugs or other oddities in how well programs handle **environ.

$ unset FOO
$ ./envdup perl -e 'exec "env"' | grep FOO
FOO=bar
$ ./envdup python3 -c 'import os;os.execvp("env",["env"])' | grep FOO
FOO=bar
FOO=zot
$ 

Looks like Python 3.6 here blindly passes along the duplicates (a leaky abstraction) while Perl 5.24 does not. How about the shells?

$ ./envdup bash -c 'echo $FOO; exec env' | egrep 'bar|zot'
zot
FOO=zot
$ ./envdup zsh -c 'echo $FOO; exec env' | egrep 'bar|zot' 
bar
FOO=bar
$ 

Gosh, what happens if sudo only sanitizes the first environment entry but then bash runs with the second? Hello PATH or LD_RUN_PATH exploit. Is your sudo (and everything else?) patched for that hole? Security exploits are neither "an anecdotal difference" nor just "a bug" in the calling program.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.