10

Is there a way to source a shell script into a namespace, preferably a bash shell script but I would look into other shells if they had this feature and bash didn't.

What I mean by that is, e.g., something like "prefix all defined symbols with something so that they don't collide with already defined symbols (variable names, function names, aliases)" or any other facility that prevents name collisions.

If there's a solution where I can namespace at source time (NodeJS style), that would be the best.

Example code:

$ echo 'hi(){ echo Hello, world; }' > english.sh
$ echo 'hi(){ echo Ahoj, světe; }' > czech.sh
$ . english.sh
$ hi
 #=> Hello, world
$ . czech.sh #bash doesn't even warn me that `hi` is being overwritten here
$ hi
 #=> Ahoj, světe
#Can't use the English hi now
#And sourcing the appropriate file before each invocation wouldn't be very efficient 
4
  • 1
    Thanks for the clarification. I expect that the answer is negative. The usual shell programming paradigm is that when you want to isolate changes, you do it in a subshell, which creating one is just about the ( easiest thing ever ). But that's not quite what you're after. I guess you could do ( stuff in subshell; exec env ) | sed 's/^/namespace_/' and eval the result in the parent shell but that's kind of nasty.
    – Celada
    Jul 5 '15 at 12:15
  • 3
    Yes. Get ksh93. Namespaces are fundamental to it - and all of its types of names (which are also typeable) support namespacing. It's also much faster in practically every respect than bash, by the way.
    – mikeserv
    Jul 5 '15 at 12:40
  • @mikeserv Thanks, if you add it as an answer with a code example that demonstrates the functionality, I'll accept it.
    – PSkocik
    Jul 5 '15 at 13:07
  • @michas I would need to namespace function symbols and aliases too. env | sed ... would work for variables, I could do set to get functions, but the search and replace would be a problem -- functions can call each other and you so you'd need to replace the all the cross invocations with prefixed cross invocations but without replacing the same words elsewhere in the function definition code, where it isn't an invocation. For that you'd need a bash parser, not just a regex, and it would still only work as long as the functions didn't call each other via eval.
    – PSkocik
    Jul 5 '15 at 13:16
11

From man ksh on a system with a ksh93 installed...

  • Name Spaces
    • Commands and functions that are executed as part of the list of a namespace command that modify variables or create new ones, create a new variable whose name is the name of the name space as given by identifier preceded by .. When a variable whose name is name is referenced, it is first searched for using .identifier.name.
    • Similarly, a function defined by a command in the namespace list is created using the name space name preceded by a ..
    • When the list of a namespace command contains a namespace command, the names of variables and functions that are created consist of the variable or function name preceded by the list of identifiers each preceded by .. Outside of a name space, a variable or function created inside a name space can be referenced by preceding it with the name space name.
    • By default, variables staring with .sh are in the sh name space.

And, to demonstrate, here is the concept applied to a namespace provided by default for every regular shell variable assigned in a ksh93 shell. In the following example I will define a discipline function that will act as the assigned .get method for the $PS1 shell variable. Every shell variable basically gets its own namespace with, at least, the default get, set, append, and unset methods. After defining the following function, any time the variable $PS1 is referenced in the shell, the output of date will be drawn at the top of the screen...

function PS1.get {
    printf "\0337\33[H\33[K%s\0338" "${ date; }"
}

(Also note the lack of the () subshell in the above command substitution)

Technically, namespaces and disciplines are not exactly the same thing (because disciplines can be defined to apply either globally or locally to a particular namespace), but they are both part and parcel to the conceptualization of shell data types which is fundamental to ksh93.

To address your particular examples:

echo 'function hi { echo Ahoj, světe\!;  }' >  czech.ksh
echo 'function hi { echo Hello, World\!; }' >english.ksh
namespace english { . ./english.ksh; }
namespace czech   { . ./czech.ksh;   }
.english.hi; .czech.hi

Hello, World!
Ahoj, světe!

...or...

for ns in czech english
do  ".$ns.hi"
done

Ahoj, světe!
Hello, World!
1
  • @PSkocik - why didn't you fix my ehoj thing? I could have sworn that's what it said before... sorry about that. I wouldn't have accepted an answer written by someone who didn't even bother to spell the words I used in the question correctly... Honestly, though, I think I remember just copy/pasting jt... hmm...
    – mikeserv
    Jul 5 '15 at 17:30
3

I have written a POSIX shell function which might be used to locally namespace a shell builtin or function in any of ksh93, dash, mksh, or bash (named specifically because I have personally confirmed it to work in all of these). Of the shells in which I tested it, it only failed to meet my expectations in yash, and I never expected it to work at all in zsh. I did not test posh. I gave up any hope for posh a while ago and haven't installed it in some time. Maybe it works in posh...?

I say it is POSIX because, by my reading of the specification, it takes advantage of a specified behavior of a basic utility, but, admittedly, the specification is vague in this regard, and, at least one person apparently disagrees with me. Generally I have had a disagreement with this one I have eventually found the error to be my own, and possibly I am also wrong this time about the specification as well, but when I questioned him further he did not reply.

As I said, though, this definitely does work in the aforementioned shells, and it works, basically, in the following way:

some_fn(){ x=3; echo "$x"; }
x=
x=local command eval some_fn
echo "${x:-empty}"

3
empty

The command command is specified as a basically available utility and one of the pre-$PATH'd builtins. One of its specified functions is to wrap special builtin utilities in its own environment when calling them, and so...

{       sh -c ' x=5 set --; echo "$x"
                x=6 command set --; echo "$x"
                exec <"";  echo uh_oh'
        sh -c ' command exec <""; echo still here'
}

5
5
sh: 3: cannot open : No such file
sh: 1: cannot open : No such file
still here

...the behavior of both command-line assignments above is correct by spec. The behavior of both error conditions is also correct, and is in fact very nearly completely duplicated there from the specification. Assignments prefixed to the command-lines of functions or special builtins are specified to affect the current shell environment. Likewise, redirection errors are specified as fatal when pointed at either of those. command is specified to suppress the special treatment of special builtins in those cases, and the redirection case is actually demonstrated by example in the specification.

Regular builtins, like command, on the other hand, are specified to run in a subshell environment - which doesn't necessarily mean that of another process, just that it should be fundamentally indistinguishable from one. The results of calling a regular builtin ought always to resemble what might be obtained from a similarly capable $PATH'd command. And so...

na=not_applicable_to_read
na= read var1 na na var2 <<"" ; echo "$var1" "$na" "$var2"
word1 other words word2

word1 not_applicable_to_read word2

But the command command cannot call shell functions, and so cannot be used to render their special treatment moot as it can for regular builtins. That's also spec'd. In fact, the spec says that a primary utility of command is that you can use it within a wrapper shell function named for another command to call that other command without self-recursion because it will not call the function. Like this:

cd(){ command cd -- "$1"; }

If you didn't use command there the cd function would almost definitely segfault for self-recursion.

But as a regular builtin which can call special builtins command can do so in a subshell environment. And so, while current shell-state defined within might stick to the current shell - certainly read's $var1 and $var2 did - at least the results of command-line defines probably should not...

Simple Commands

If no command name results, or if the command name is a special built-in or function, variable assignments shall affect the current execution environment. Otherwise, the variable assignments shall be exported for the execution environment of the command and shall not affect the current execution environment.

Now whether or not command's ability to be both a regular builtin and to directly call special builtins is just some kind of unexpected loophole with regard to command-line defines I don't know, but I do know that at least the four shells already mentioned honor the command namespace.

And though command cannot directly call shell functions, it can call eval as demonstrated, and so can do so indirectly. So I built a namespace wrapper on this concept. It takes a list of arguments like:

ns any=assignments or otherwise=valid names which are not a command then all of its args

...except that the command word above is only recognized as one if it can be found with an empty $PATH. Besides locally-scoping shell variables named on the command line, it also locally-scopes all variable with single lower-case alphabetic names and a list of other standard ones, such as $PS3, $PS4, $OPTARG, $OPTIND, $IFS, $PATH, $PWD, $OLDPWD and some others.

And yes, by locally scoping the $PWD and $OLDPWD variables and afterward explicitly cding to $OLDPWD and $PWD it can fairly reliably scope the current working directory as well. This isn't guaranteed, though it does try pretty hard. It retains a descriptor for 7<. and when its wrap target returns it does cd -P /dev/fd/7/. If the current working directory has been unlink()'d in the interim it should still at least manage to change back into it but will emit an ugly error in that case though. And because it maintains the descriptor, I don't think a sane kernel should allow its root device to be unmounted either (???).

It also locally scopes shell options, and restores these to the state in which it found them when its wrapped utility returns. It treats $OPTS specially in that it maintains a copy in its own scope which it initially assigns the value of $-. After also handling all assignments on the command-line it will do set -$OPTS just before invoking its wrap target. In this way if you define -$OPTS on the command-line you can define a your wrap target's shell options. When the target returns it will set +$- -$OPTS with its own copy of $OPTS (which is not affected by the command-line defines) and restore all to the original state.

Of course, there's nothing stopping the caller from somehow explicitly returrning out of the function by way of the wrap target or its arguments. Doing so will prevent any state restoration/cleanup it would otherwise attempt.

To do all that it needs to go three eval's deep. First it wraps itself in a local scope, then, from within, it reads in arguments, validates them for valid shell names, and quits with error if it finds one which isn't. If all arguments are valid and eventually one causes command -v "$1" to return true (recall: $PATH is empty at this point) it will eval the command-line defines and pass off all remaining arguments to the wrap target (though it ignores the special case for ns - because that wouldn't be very useful, and three evals deep is more than deep enough).

It basically works like this:

case $- in (*c*) ... # because set -c doesnt work
esac
_PATH=$PATH PATH= OPTS=$- some=vars \
    command eval LOCALS=${list_of_LOCALS}'
        for a do  i=$((i+1))          # arg ref
              if  [ "$a" != ns ]  &&  # ns ns would be silly
                  command -v "$a" &&
              !   alias "$a"          # aliases are hard to run quoted
        then  eval " PATH=\$_PATH OTHERS=$DEFAULTS $v \
                     command eval '\''
                             shift $((i-1))         # leave only tgt in @
                             case $OPTS in (*different*)
                                  set \"-\${OPTS}\" # init shell opts 
                             esac
                             \"\$@\"                # run simple command
                             set +$- -$OPTS "$?"    # save return, restore opts
                     '\''"
              cd -P /dev/fd/7/        # go whence we came
              return  "$(($??$?:$1))" # return >0 for cd else $1
        else  case $a in (*badname*) : get mad;;
              # rest of arg sa${v}es
              esac
        fi;   done
    ' 7<.

There are some other redirections and, and a few weird tests to do with the way some shells put c in $- and then refuse to accept it as an option to set (???), but its all ancillary, and primarily used just to save from emitting unwanted output and similar in edge cases. And that's how it works. It can do those things because it sets up its own local scope before calling its wrapped utility in a nested such.

It's long, because I try to be very careful here - three evals is hard. But with it you can do:

ns X=local . /dev/fd/0 <<""; echo "$X" "$Y"
X=still_local
Y=global
echo "$X" "$Y"

still_local global
 global

Taking that a step further and persistently namespacing the local scope of the wrapped utility shouldn't be very difficult. And even as written it already defines a $LOCALS variable to the wrapped utility which is comprised of only a space separated list of all names that it defined in the wrapped utility's environment.

Like:

ns var1=something var2= eval ' printf "%-10s%-10s%-10s%s\n" $LOCALS '

...which is perfectly safe - $IFS has been sanitized to its default value and only valid shell names make it into $LOCALS unless you set it yourself on the command line. And even if there might be glob characters in a split variable, you could set OPTS=f on the command-line as well for the wrapped utility to prohibit their expansion. In any case:

LOCALS    ARG0      ARGC      HOME
IFS       OLDPWD    OPTARG    OPTIND
OPTS      PATH      PS3       PS4
PWD       a         b         c
d         e         f         g
h         i         j         k
l         m         n         o
p         q         r         s
t         u         v         w
x         y         z         _
bel       bs        cr        esc
ht        ff        lf        vt
lb        dq        ds        rb
sq        var1      var2      

And here's the function. All of the commands are prefixed w/ \ to avoid alias expansions:

ns(){  ${1+":"} return
       case  $- in
       (c|"") ! set "OPTS=" "$@"
;;     (*c*)  ! set "OPTS=${-%c*}${-#*c}" "$@"
;;     (*)      set "OPTS=$-" "$@"
;;     esac
       OPTS=${1#*=} _PATH=$PATH PATH= LOCALS=     lf='
'      rb=\} sq=\' l= a= i=0 v= __=$_ IFS="       ""
"      command eval  LOCALS=\"LOCALS \
                     ARG0 ARGC HOME IFS OLDPWD OPTARG OPTIND OPTS     \
                     PATH PS3 PS4 PWD a b c d e f g h i j k l m n     \
                     o p q r s t u v w x y z _ bel bs cr esc ht ff    \
                     lf vt lb dq ds rb sq'"
       for a  do     i=$((i+1))
              if     \[ ns != "$a" ]         &&
                     \command -v "$a"  >&9   &&
              !      \alias "${a%%=*}" >&9 2>&9
              then   \eval 7>&- '\'    \
                     'ARGC=$((-i+$#))  ARG0=$a      HOME=~'           \
                     'OLDPWD=$OLDPWD   PATH=$_PATH  IFS=$IFS'         \
                     'OPTARG=$OPTARG   PWD=$PWD     OPTIND=1'         \
                     'PS3=$PS3 _=$__   PS4=$PS4     LOCALS=$LOCALS'   \
                     'a= b= c= d= e= f= g= i=0 j= k= l= m= n= o='     \
                     'p= q= r= s= t= u= v= w= x=0 y= z= ht=\   '      \
                     'cr=^M bs=^H ff=^L vt=^K esc=^[ bel=^G lf=$lf'   \
                     'dq=\" sq=$sq ds=$ lb=\{ rb=\}' \''"$v'          \
                            '\command eval       9>&2 2>&- '\'        \
                                   '\shift $((i-1));'                 \
                                   'case \${OPTS##*[!A-Za-z]*} in'    \
                                   '(*[!c$OPTS]*) >&- 2>&9"'\'        \
                                   '\set -"${OPTS%c*}${OPTS#*c}"'     \
                                   ';;esac; "$@" 2>&9 9>&-; PS4= '    \
                                   '\set  +"${-%c*}${-#*c}"'\'\"      \
                                          -'$OPTS \"\$?\"$sq";'       \
              '             \cd -- "${OLDPWD:-$PWD}"
                            \cd -P  ${ANDROID_SYSTEM+"/proc/self/fd/7"} /dev/fd/7/
                            \return "$(($??$?:$1))"
              else   case   ${a%%=*}      in
                     ([0-9]*|""|*[!_[:alnum:]]*)
                            \printf "%s: \${$i}: Invalid name: %s\n" \
                            >&2    "$0: ns()"   "'\''${a%%=*}'\''"
                            \return 2
              ;;     ("$a") v="$v $a=\$$a"
              ;;     (*)    v="$v ${a%%=*}=\${$i#*=}"
              ;;     esac
                     case " $LOCALS " in (*" ${a%%=*} "*)
              ;;     (*)    LOCALS=$LOCALS" ${a%%=*}"
              ;;     esac
              fi
       done'  7<.    9<>/dev/null
}
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.