2

On Android (which uses mksh, MirBSD Korn Shell), there's a special syntax of string substitution (called "value substitution"):

${|commands}

Instead of collecting output of the commands (as with `` and $()), the substitution result is taken from the $REPLY variable assigned inside the delimiters. It's special in that commands do NOT run in a subshell - they run in the same shell and has access to everything the current shell session has.

Debian has a mksh package for MirBSD Korn Shell, which acts exactly like that of Android.

What shells support a similar syntax, in that:

  • Commands run in the same shell as the parent
  • Substitution result is taken from other ways than file descriptors
    • It should be easy to note that reading outputs or pipes counts as FDs
2

To answer the question that is actually asked: None. No other shells have this.

As the MirBSD Korn shell changelog tells us, value substitutions were added by Thomas Goirand in 2014 to release 46.

No other shell has then or since copied this idea, to my knowledge. Some of them have the thing that value substitutions were derived from, but they do not actually have value substitutions.

  • Note that zsh's math functions and dynamic named directories are similar features. See also the ${ ...; } form of command substitution copied by mksh from ksh93, and fish's command substitution which doesn't create a subshell environment. Related: zsh.org/mla/workers/2019/msg00775.html – Stéphane Chazelas Sep 24 '20 at 8:18
  • That's the thing that value substitutions were derived from. It is not value substitutions. And a markedly different syntax is not the "similar syntax" that the question asked for. As I said, this is answering the question actually asked. – JdeBP Sep 24 '20 at 8:22
  • What changelog are you referring to? That valsub feature was committed to mksh repository on 2013-05-02 (with Thorsten Glaser as author) and released in R46 on 2013-05-03. I can't find any evidence of a Thomas Goirand ever writing code for mksh. – Stéphane Chazelas Sep 25 '20 at 14:04
1

The ${ cmds;} form of the command substitution of ksh93 runs cmds in the same shell, but otherwise captures the stdout like a regular command substitution. Example:

a=1; echo ${ a=2; echo wtf;}; echo $a
wtf
2

And the fact that it captures the stdout of the commands is exactly what makes it useful, since you don't have to save the output into some temporary file and then read it back, or set up a named pipe, or rewrite some hairy function to make it append its output to some variable instead of just writing it out.

And that's very much unlike that "value substitution" mksh feature, for which I'm unable to find any rationale. Why can't you assign the REPLY variable before, and then just use it as $REPLY?

  • Note that ${ ...; } (also added to mksh) like $(...) can't be used to return arbitrary values as it strips all trailing newline characters. The ${| ...;} variant doesn't have the problem and is also more efficient as it avoids storing the output in a temp file (or rewriting the whole shell infrastructure à la ksh93, which is still very buggy on that front. ksh93v-/2020 (now abandoned) have tried to fix some of those bugs by reverting to temp files like in mksh, but it comes with its own bug like this one which also affects mksh. – Stéphane Chazelas Sep 24 '20 at 6:18
  • Doesn't a regular subshell $(...) also strip trailing newlines? – user414777 Sep 24 '20 at 6:51
  • Yes, I did say like $(...). That's a common problem with command substitution, you'll find plenty of Q&As here discussing how to work around it. – Stéphane Chazelas Sep 24 '20 at 6:58
  • @StéphaneChazelas sorry, missed it. FWIW, the ksh 93u+ doesn't exhibit that bug, and it's using a temporary file to implement ${ ...;}. ksh93 -c 'echo ${ ls -l /dev/fd/1;}' => /tmp/sf3t.bnk (deleted). – user414777 Sep 24 '20 at 7:02
  • 1
0

There is a description of how that expansion is supposed to work and is useful in this page

However: I fail to see how that substitution is actually useful.

This two lines are different only in the additional character |:

$ mksh -c '
    val=one; echo $val; 
    REPLY=init; 
    thing="${|REPLY=beep; other=tested;}"; 
    echo "thing=$thing"; echo "val=$val"; 
    echo "REPLY=$REPLY"; echo "other=$other"
 '

one
thing=beep
val=one
REPLY=init
other=tested

$ mksh -c '
    val=one; echo $val; 
    REPLY=init; 
    thing="${ REPLY=beep; other=tested;}"; 
    echo "thing=$thing"; echo "val=$val"; 
    echo "REPLY=$REPLY"; echo "other=$other"
 '

one
thing=
val=one
REPLY=beep
other=tested

And the only difference in the results are the values of thing and REPLY.

The simplest alternative that has the exact same effects as the mksh's ${| is (a simplified alternative of mksh's : ${ thing=beep; other=tested; }; ):

$ mksh -c '
    val=one; echo $val; 
    REPLY=init; 
    { thing=beep; other=tested; };
    echo "thing=$thing"; echo "val=$val";
    echo "REPLY=$REPLY"; echo "other=$other"
 '

one
thing=beep
val=one
REPLY=init
other=tested

In words:

Just assign the value to the variable wanted (thing) instead of REPLY.

That works in all shells that allow the use of {…} (which is almost all).

Simpler example (works in ksh,mksh,zsh,bash):

{ thing=$(date); }
echo "$thing"
0

That feature, called value substitution or valsub by Thorsten Glaser (aka @mirabilos), the maintainer or MirBSD and its shell mksh (derived from pdksh), is specific to mksh.

It was committed to the mksh code base on May 2nd 2013, and released in version R46 announced the next day on the mksh mailing list.

It's written on the back of the ${ body; } form of command substitution (called function substitution or funsub in mksh) copied from ksh93 in February that same year and also released in R46.

In ksh93, I/O from builtins is virtualised. Neither $(builin-cmd) nor ${ builtin-cmd; } involve any fork or I/O. So $(print foo) or ${ print foo; } expand to foo and meet your requirements of an operator that doesn't involve any fd.

In both forms, the print builtin doesn't write anything to any fd, but its would-be-output (trimmed of trailing newline characters) makes up the expansion. The difference between the two is that $(...) introduces a subshell environment (which contrary to other shells is not implemented by forking a child process), while ${ ...; } doesn't.

Now, to be able do that, ksh93 (a rewrite of ksh (itself from 1983) almost from scratch), all the I/O in the shell had to be rewritten specifically. When mksh added that ${ ...; } feature in 2013, it took a simpler approach which just records the output in a deleted temporary file, and reads that file's contents after the code in it returns to make up the expansion.

That means however that the output ends up stored on disk even if temporarily and the I/O means poorer performance than if the resulting data was just passed around in memory like in ksh93. So I suppose that's why Thorsten added that ${| ...; } separate form which can pass the value using a dedicated variable ($REPLY), and again doesn't require a major rework of the shell internals.

That means however that functions that are used in that fashion have to written specifically to return their value in $REPLY (which can only be scalar not a list except via split+glob), and just becomes a bit of syntactic sugar. Example:

sanitize() {
  REPLY=${1//[!0123456789-]}
  local sign=
  case $REPLY in
    (-*) REPLY=${REPLY#-}; sign=-
  esac
  REPLY=$sign${REPLY//-}
}

print "$(( ${|sanitize "$1"} + ${|sanitize "$2"} ))"

Without it, you'd have to write:

sanitize "$1"; a=$REPLY
sanitize "$2"; b=$REPLY
print "$(( a + b ))"

One advantage over $(...) and ${ ...; } is that it doesn't strip trailing newline characters. For instance, $(basename -- "$file") is wrong as it doesn't work if $file ends in newline characters, while ${|basename -- "$file"} (assuming basename was rewritten as a function that returns the base name in $REPLY) wouldn't have the problem.

Other shells with constructs that can return values without involving I/O:

zsh

Someone did propose to implement a simplified version of mksh's valsub in 2019 which eventually evolved to this proposal, but as far as I know, it has not made it to zsh yet.

However, zsh has several alternative ways to have expansions being the result of arbitrary code without involving a subshell nor I/O.

Math functions

For arithmetic, zsh has the concept of math functions:

square() (( $1 * $1))
functions -M square 1

echo $(( square(5) + square(12) ))

That's limited to numbers though (integer or float) and can only be used in arithmetic expressions. The math functions themselves though can take non-numbers as arguments with functions -sM), so though very convoluted, you could do:

func() REPLY=foo$1; functions -sM func

echo ${$((func(bar)))+$REPLY} ${$((func(baz)))+$REPLY}

as an equivalent of mksh's:

func() REPLY=foo$1

echo "${|func bar}" "${|func baz}"

Dynamic named directories

zsh has another form of expansion than can be computed with shell code without I/O. That's using a customisation framework for tilde expansion called Dynamic named directories (see info zsh dynamic).

If you define:

autoload -Uz add-zsh-hook

valsub() {
  [[ $1 = n && $2 = '!'* ]] && eval "${2#?}" && reply=("$REPLY")
}
add-zsh-hook -Uz zsh_directory_name valsub

Then a tilde expansion of the form ~[!'REPLY=something'] would expand to something.

Tilde expansion is not done in every context, but you could also use that dynamic named directories feature as part of parameter expansion using that kind of trick described at that discussion about valsub support mentioned above.

e and + glob qualifiers

Globs can expand to the result of arbitrary code as well using the e (for evaluation) or + glob qualifiers.

Those are normally used to filter files based on the result of some code.

Like:

ls -ld -- *.txt(e['(( $#REPLY > 20 ))'])

To select txt filenames those length is greater than 20 characters. But can also be used to change the result of the expansion:

ls -ld -- *.txt(e['REPLY=$REPLY:r.html'])

(expand to txt files with the extension replaced with html). Or even:

ls -ld -- *.txt(e['reply+=($REPLY:r.html)'])

Returns both txt and html translation.

So you can actually do:

echo /(e['REPLY=foobar'])

For that to expand to the result of arbitrary code, here applying the qualifier to / which we know always exists. Or even a list:

printf '<%s>\n' /(e['reply=(foo bar)'])

The + qualifier is a variant that just takes a function name, so you can do echo /(+func) where func is the function that generates the expansion.

Again, like for ~ expansion, globbing is not done in every context.

es

es is a derivative of Byron Rakitzis's public domain clone of the Research Unix V10/Plan9 rc shell.

rc's functions can return a list of exit statuses (can be signal name or positive integers) and is made available to the caller in the $status list variable.

es extended it to be able to return any list of anything and instead of making it available in $status, the exit status (or function returned value) is obtained with the <={...} syntax.

So you can do:

fn foo { return foo$1 }
echo <={foo bar}

for instance.

Note however that only a returned value made of an empty list or a list whose elements are all empty or 0 is interpreted as being successful. So, for instance, here foo anything && echo bar would never output bar as foo always returns a value that is never interpreted as success.

ksh93

Beside $(...), ${ ...; } already discussed, there are a features that allow expansions to have dynamic contents without involving I/O:

disciplines

You can define a function that is invoked every time a variable is set or expanded. For associative array variables, those functions will have access to the subscript, so you could use that to pass one arbitrary argument to the function:

typeset -A valsub
function valsub.get {
  .sh.value=foo${.sh.subscript}
}
echo "${valsub[bar]}"

would output foobar.

math functions

ksh93 also has math functions, though with a different syntax from those of zsh:

function .sh.math.square x {((.sh.value = x*x))}
echo "$(( square(5) + square(12) ))"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.