Setting variables in /etc/environment not having an affect but setting them in command line is [duplicate]

Question

I am setting certain environment variables for the command pyspark to work. When I set the variables in /etc/environment and source it, it doesn't work. However, when I set them in command line they do work but ofcourse only for this session. My intent is to set them globally so that even if I re-open the session I can just type pyspark

Setting in /etc/environment

[root@localhost ~]# more /etc/environment
[root@localhost ~]# echo "export SPARK_HOME=/srv/spark" >> /etc/environment
[root@localhost ~]# echo "export PATH="$SPARK_HOME"/bin:"$PATH >> /etc/environment
[root@localhost ~]# echo "export JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk" >> /etc/environment
[root@localhost ~]# source /etc/environment
[root@localhost ~]# pyspark --version
-bash: pyspark: command not found

Setting on command line

[root@localhost ~]# export SPARK_HOME=/srv/spark
[root@localhost ~]# export PATH=$SPARK_HOME/bin:$PATH
[root@localhost ~]# export JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk
[root@localhost ~]# pyspark --version
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.6.1
      /_/

Type --help for more information.

Answer 1

Put the export SPARK_HOME=... etc. commands in the startup files of your shell. With bash, that would be either ~/.profile or ~/.bash_profile.

---

On Linux, /etc/environment is usually read by pam_env.so during login, and it doesn't support expanding existing variables, so setting PATH=$PATH:/something will result in the literal string $PATH to appear in your PATH. This isn't what you want. (See e.g. this and this, also for fun this.)

Also, setting PATH in /etc/environment might not work, since the global startup scripts for the shell might rewrite them. (They do on Debian by default, on the old CentOS I have handy, the startup scripts only seem to prepend to PATH).

If your system doesn't use pam_env.so, but you only source the script by hand, then these considerations don't matter, of course. But it looks like it's widely used by at least a couple of Linux distributions, so it might be a good idea to use another filename.

---

(Because this is completely opposite to what the other answers said, I tested it on an old CentOS.)

I put the following in /etc/environment:

export FOO1=bar
export FOO2=foo:$FOO

After logging in again, set | grep FOO shows:

FOO1=bar
FOO2='foo:$FOO'

Answer 2

This line doesn't do what you think:

echo "export PATH="$SPARK_HOME"/bin:"$PATH >> /etc/environment

The problem is that $SPARK_HOME is being evaluated during the echo (as is $PATH; you probably don't want your current PATH in that file).

If you cat /etc/environment you'll see the line doesn't look right.

Instead, use

echo 'export PATH="$SPARK_HOME/bin:$PATH"' >> /etc/environment

This will use the values of $SPARK_HOME and $PATH in place when you source the file.