9

I have the following recursive function to set environment variables:

function par_set {
  PAR=$1
  VAL=$2
  if [ "" != "$1" ]
  then
    export ${PAR}=${VAL}
    echo ${PAR}=${VAL}
    shift
    shift
    par_set $*
  fi
}

If I call it by itself, it both sets the variable and echoes to stdout:

$ par_set FN WORKS
FN=WORKS
$ echo "FN = "$FN
FN = WORKS

Redirecting stdout to a file also works:

$ par_set REDIR WORKS > out
cat out
REDIR=WORKS
$ echo "REDIR = "$REDIR
REDIR = WORKS

But, if I pipe stdout to another command, the variable doesn't get set:

$ par_set PIPE FAILS |sed -e's/FAILS/BARFS/'
PIPE=BARFS
$ echo "PIPE = "$PIPE
PIPE =

Why does the pipe prevent the function from exporting the variable? Is there a way to fix this without resorting to temp files or named pipes?

Solved:

Working code thanks to Gilles:

par_set $(echo $*|tr '=' ' ') > >(sed -e's/^/  /' >> ${LOG})

This allows the script to be called thusly:

$ . ./script.sh PROCESS_SUB ROCKS PIPELINES=NOGOOD
$ echo $PROCESS_SUB
ROCKS
$ echo $PIPELINES
NOGOOD
$ cat log
7:20140606155622162731431:script.sh:29581:Parse Command Line parameters.  Params must be in matched pairs separated by one or more '=' or ' '.
  PROCESS_SUB=ROCKS
  PIPELINES=NOGOOD

Project hosted on bitbucket https://bitbucket.org/adalby/monitor-bash if interested in full code.

8

Each part of a pipeline (i.e. each side of the pipe) runs in a separate process (called a subshell, when a shell forks a subprocess to run part of the script). In par_set PIPE FAILS |sed -e's/FAILS/BARFS/', the PIPE variable is set in the subprocess that executes the left-hand side of the pipe. This change is not reflected in the parent process (environment variables do not transfer between processes, they are only inherited by subprocesses.

The left-hand side of a pipe always runs in a subshell. Some shells (ATT ksh, zsh) run the right-hand side in the parent shells; most also run the right-hand side in a subshell.

If you want to both redirect the output of a part of the script and run that part in the parent shell, in ksh/bash/zsh, you can use process substitution.

par_set PROCESS SUBSTITUTION > >(sed s/ION/ED/)

With any POSIX shell, you can redirect the output to a named pipe.

mkfifo f
<f grep NAMED= &
par_set NAMED PIPE >f

Oh, and you're missing quotes around variable substitutions, your code breaks on things like par_set name 'value with spaces' star '*'.

export "${PAR}=${VAL}"
…
par_set "$@"
  • Process substitution for the win! I knew I could use a named pipe or temp file, but those are ugly, have poor concurrency, and leave a mess behind if the script dies (trap helps with the last one). The space thing is intentional. By convention, variables passed on the command line are in name/value pairs and separated by '=' and/or ' '. – Andrew Jun 6 '14 at 15:45
3

This doesn't work because each side of the pipe runs in a subshell in bash, and variables set in a subshell are local to that subshell.

Update:

It looks like it's easy to pass variables from the parent to the child shell, but really hard to do it the other way. Some workarounds are named pipes, temp files, writing to stdout and reading in the parent, etc.

Some references:

http://mywiki.wooledge.org/BashFAQ/024
https://stackoverflow.com/q/15541321/3565972
https://stackoverflow.com/a/15383353/3565972
http://forums.opensuse.org/showthread.php/458979-How-export-variable-in-subshell-back-out-to-parent

  • I thought that was a possibility, but the function should execute in the current shell. I tested by adding an "echo $$" to the function. par_set STILL FAILS|sed -e"s/^/sedpid = $$, fnpid = /" outputs sedpid = 15957, fnpid = 15957. – Andrew Jun 5 '14 at 19:49
  • @Andrew See this stackoverflow.com/a/20726041/3565972 . Apparently, $$ is the same for both parent and child shells. You can use $BASHPID to get the subshell pid. When I echo $$ $BASHPID within par_set I get different pids. – savanto Jun 5 '14 at 20:25
  • @Andrew Still trying to find a workaround, but failing! =) – savanto Jun 5 '14 at 20:26
  • @Savanto-Thanks. I did not know that about $$ vs $BASHPID or the pipe forcing subshells. – Andrew Jun 5 '14 at 21:21
0

You point out the subshells - that can be got around with some finagling in the shell outside a pipeline - but the harder part of the problem has to do with the pipeline's concurrency.

All process members of the pipeline start at once, and so the problem might be easier to understand if you look at it like this:

{ sleep 1 ; echo $((f=1+2)) >&2 ; } | echo $((f))
###OUTPUT
0
...
3

The pipeline processes can't inherit the variable values because they're already off and running before the variable is ever set.

I can't really understand what the point of your function is - what purpose does it serve that export doesn't already? Or even just var=val? For instance, here's almost the same pipeline again:

pipeline() { 
    { sleep 1
      echo "f=$((f=f+1+2))" >&3
    } | echo $((f)) >&2
} 3>&1

f=4 pipeline

###OUTPUT

4
...
f=7

And with export:

export $(f=4 pipeline) ; pipeline

###OUTPUT:

4
7
...
f=10

So your thing could work like:

par_set $(echo PIPE FAILS | 
    sed 's/FAIL/WORK/;/./w /path/to/log')

Which would log to a file sed's output and deliver it to your function as a shell-split "$@".

Or, alternatively:

$ export $(echo PIPE FAILS | sed 's/ FAIL/=WORK/')
$ par_set $PIPE TWICE_REMOVED
$ echo "WORKS = "$WORKS
WORKS = TWICE_REMOVED

If I were going to write your function, though, it would probably look like this:

_env_eval() { 
    while ${2+:} false ; do
       ${1:+export} ${1%%["${IFS}"]*}="${2}" || :
       shift 2
    done
}
  • That's true, but irrelevant: Andrew is trying to use the variables after the pipeline ends, not on the other side of the pipe. – Gilles Jun 5 '14 at 23:45
  • @Gilles - well he can do that. |pipeline | sh – mikeserv Jun 6 '14 at 0:18
  • @Gilles - actually, you don't even need sh. He's already using export. – mikeserv Jun 6 '14 at 3:23
  • Overall goal is system monitoring scripts. Would like to be able to call them interactively, from other scripts, or from cron. Want to be able to pass parameters by setting env vars, passing on command line, or config file. Function exists to take input from one or more sources and set the environment correctly. However, I also want to be able to optionally log the output (after running through sed to format), hence the need to pipe. – Andrew Jun 6 '14 at 5:20
  • 1
    @mikeserv- Thanks for all the comments. I went with Gilles suggestion of process substitution, but having to argue for my code makes me a better programmer. – Andrew Jun 6 '14 at 16:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.