how to share environment variables between shells like globals related to a master pid?

Question

So, I need to share environment variables between shells, and that the changes on the variables be promptly recognized by all shells, like global variables. But they must be related to a master PID, so I can have 2 shell scripts running, each being a master with same global variable name, and it children will only change the variable related to their master PID.

I use bash for scripting.

My current researches are these:

Modules
I read I could use modules project for that: http://modules.sourceforge.net/, but I am having huge trouble trying to do it, it seems to lack examples? also "modules" is a too generic word, very hard to google about because mixes with many other things… after some weeks my hands are still empty…

Simple text source file
I read also I should use a simple text file to store/update/read variables from, as a source script file, using flock too to avoid concurrent writing. But there are problems related to doing that, like concurrent change of variable simultaneously by 2 shells.

Alternatively
I am aware that after a "global" variable is read by the shell, it will keep that value until another read is made; and another shell may change it in the mean while too…
May be a very fast external application could provide such storage and we would not try to use environment variables as globals then? but my guess is any SQL solution will be too slow…

Answer 1

You can write the environment to a file with export -p and read it back periodically in the other instances with . (the include command). But as you note, if there are concurrent modifications, the result won't be pretty.

If you want modifications to propagate instantly, you'll have to use a wrapper around the export builtin that also propagates the changes to the other shells. On Linux, you can use the flock utility.

global_export () {
  {
    flock 0
    . /path/to/env
    [ $# -eq 0 ] || export "$@"
    export -p >/path/to/env
  } </path/to/env
}

Note that you'll need to call global_export whenever you assign a new value to an exported variable.

This is rather clumsy. It's likely that there's a better way to solve your problem. The most obvious way is to have whatever command is using those environment variable read a configuration files instead.

Answer 2

In addition to just using flock from Giles' answer, I'd do something similar to the following:

Register a signal handler to reload the file:

reload_env() {
 flock -s 3
 . /path/to/env
 flock -u 3
}
trap reload_env USR1

A function to write out the file, and send a SIGUSR1 to all the shells notifying them to reload the file:

export_env() {
 flock 3
 export -p >/path/to/env
 flock -u 3
 fuser -k -USR1 /path/to/env
}

Open the file, but do nothing with it. This is just so that the fuser will be able to signal this shell:

exec 3>>/path/to/env

---

The order is somewhat important. At least the exec ... part must be after the trap ... part. Otherwise we could open the file and then get signaled by the fuser call before we've registered our signal handler.

NOTE: Code is completely untested, but the principle stands.
NOTE2: This example code is targeted at bash.

Answer 3

With the fish shell, use universally scoped variables:

set -U var value

Of course, that only works across fish shells run by the same user.

If you want to tie it to a master pid, include it in the variable name

set -U m${masterpid}var value

Answer 4

I would like to point you to named pipes for this purpose. Here is one neat example to consider.

Named pipes are created by the command:

mkfifo pipe_name

You can simply write to it by:

echo "This text goes to the pipe" > pipe_name

On the other hand, you can read from the pipe as:

read line < pipe_name;
echo $line

You can create several pipes based on how many shells do you want them to communicate. The good things about pipes is that they are not just variables. You can actually pass any data between the processes. You can pass whole files, lines of texts, whatever you wish. I believe, with this in mind, you can simplify your scripts a lot. I can imagine, if your variables point to a location of data, or a file, and the other script will have to read or retrieve it, you can directly pass the data/file without even needing a variable.

Answer 5

If ksh93 is an option, there is definitely a way to implement what you want with getter and setter discipline functions that would retrieve/store the variable value from/to a shared storage.

See https://stackoverflow.com/questions/13726293/environment-variables-to-be-used-across-multiple-korn-ksh93-shell-scripts-get for details

Unlike the otherwise useful fish universal variables feature, you have access to the implementation so you can restrict the value sharing to a group of processes, linked to a master pid through some mechanism in your use case, while keeping a single name to the variable.

Answer 6

Taking into account your comments, that you want basic IPC between shell scripts. I would just write to files (with a rename for atomicity)

varf=$(mktemp ./myvars.XXXXXX)
echo myvar=newval > $varf
mv -T -- $varf ./myvars

And have readers source ./myvars periodically.

IPC is much easier with, for example, Python and the multiprocessing module.

Answer 7

There is also this project ScriptEchoColor:

It can auto create, and manage for you, a DB text file. On the first run script the BD file will be created, the other scripts will symlink to that file as their own DB to manipulate variables values, and so, both can exchange information.

A simple sharing example with these files there at examples folder: secExplShareEnvVarsA.sh, secExplShareEnvVarsB.sh

And a synchronized (safe and consistent read/write) shared DB access there too: secExplSyncedShareEnvVarsA.sh, secExplSyncedShareEnvVarsB.sh