5

Considering a routine such as this one:

alpha() { echo a b c |tr ' ' '\n'; }

which outputs a stream, I would like to take the output stream, transform it, and paste it with the original output stream.

If I take use upcasing as a sample transformation, I can achieve what I want with:

$ mkfifo p1 p2
$ alpha | tee p1 >( tr a-z A-Z > p2) >/dev/null &
$ paste p1 p2
a       A
b       B
c       C

My question is, is there a better way to do this, preferably one not involving named pipes?

  • 3
    You know, I was working on a more flexible sort of process substitution imitation the other day. I think it's still open in the editor. Maybe I'll see if I can finish it right now. By the way, there is nothing wrong with named pipes. Try mkfifo pipe; { rm pipe; cat < file; } >pipe and in another terminal do cat pipe. – mikeserv Jul 25 '15 at 1:12
  • I put the pipe thing here. – mikeserv Jul 25 '15 at 4:02
6

In most unices you can reference a file descriptor by its device link.

alpha(){ echo a b c | tr \  \\n; }
alpha | { alpha |
    tr \[:lower:] \[:upper:] |
    paste - /dev/fd/9
}   9<&0

A   a
B   b
C   c

named pipes


Ok, so I did have a pretty well working version of this already. The version open in my editor (as yet unfinished) is not currently workable - I kind of tore the whole option parsing bit to pieces and only vaguely remember my intent. But this...

_p(){   : "$((_$$=0))"
    cd -P -- "${TMPDIR:-/tmp}"          &&
    while   [ -h "$$.$((_$$+=$$))" ]    ||
            [ -e "$$.$((_$$))" ]
    do :;   done                        &&
    mkfifo  -m a-w,u=rwx "$$.$((_$$))"  &&
    printf  %s "$PWD/$$.$((_$$))"       &&
    exec    >&- >&0                     &&
    case    $1 in
    (+)     shift
            eval "  rm $$.$((_$$))
                    cd - >/dev/null
            $@" < "$$.$((_$$))" &;;
    (-)     shift
            eval "  rm $$.$((_$$))
                    cd - >/dev/null
            $@" >   "$$.$((_$$))"   &;;
    (*) set --
        : "${1:?"USAGE: [-+] [cmd args...]"}"
    esac
}   <&- <>/dev/null

You'll likely notice there's some really weird stuff going on with $((_$$)). It's a kinduva a hacky way to attempt to step on the command's environment as little as I can. I try very hard here to keep all possible environment values as pristine as I might, because the point of this function is to do much the same as what process substitution does - run an arbitrary command that writes to a pipe.

cat "$(_p - echo a b c)"

a b c

Here's a simplified version of the workflow:

mkfifo pipe                         # make a pipe
printf %s\\n "$PWD/pipe"            # substitute its file name
exec <&- </dev/null >&- >/dev/null  # break cmd sub i/o
eval "rm pipe;$@" >pipe &           # eval rm; args

Because the eval command wraps all of its arguments in a single simple command, eval's stdout is the entire eval'd command group's stdout. And it is the shell which owns that pipe descriptor and not any of its child processes which merely inherit it as stdout. And so the shell does the open() - and because we do a blocking open() (like > rather than <>) the shell hangs until that pipe gets a reader.

This means that rm does not execute - cannot be executed - until some other process establishes a read file descriptor for that pipe. When some process opens it for reading (as cat does above) the shell stops hanging, and the first action that is taken is the pipe is rm'd - before anything else. This is not a problem - the pipe already has a reader process and a writer process. All is well.

It's a little buggy. For example:

echo "$(_p -)"

/tmp/6023.6023

^That pipe does not get rm'd. echo doesn't read it. So eval's still waiting. To kill it:

: </tmp/6023.6023

...is enough to kill it, though.

Stuff like this is what I use it for:

exec 9<> "$(_p -)"
sed -ue's/sed/mikeserv/' <&9 &
echo hi sed >&9

hi mikeserv

By the way, I've only shown the <() equivalent, but >() can be simulated like...

echo hi sed >"$(_p + sed -e's/sed/mikeserv/' '>&2')"

hi mikeserv

other pipes


You don't need my hacky function for this. Process substitution is pretty widespread. And there's nothing to stop you keeping your pipe when you ask for one - you needn't immediately assign it and discard it.

(Paranoid note: I kind of hate it because I never know where those files come from - chances are you're a more well-adjusted individual and so the following is acceptable to you)

eval "exec 9<> " <(:)

There's a pipe - it's all yours. There's no cleanup required. You're good to go.


_pp - hehe


_pp(){  :   "$((_$$=0))"
        _e()    case    ${1:-0} in
                (i|0)   exec >&- 1<>/dev/null;;
                (o|1) ! exec <&-  <>/dev/null;;
                (e|[2-9])
                        exec <&-  <>/dev/null >&- >&0
                        set     "${1#e}"
                        return  "${1:-2}";;
                (*[\<\>]*)
                        eval "
                        exec <&-  <>/dev/null >&- >&0 $1"
                esac
        cd -P   -- "${TMPDIR:-/tmp}"            &&
        while   [ -h "$$.$((_$$+=1))" ]         ||
                [ -e "$$.$((_$$))"    ]
        do :;   done                            && 
        mkfifo  -m a-w,u=rwx   "$$.$((_$$))"    &&
        printf  "%s/%s" "$PWD" "$$.$((_$$))"    &&
        case    $1  in
        (+*)    set \> \< "$@"  ;;
        (-*)    set \< \> "$@"  ;;
        (*)     rm  "$$.$((_$$))"
                set ''  USAGE:  '<-+>[+-[fd][file]]'
                ${1:?"$(printf "\n%s\t%s [cmd...]$@")"};;
        esac    &&
        case    $3  in
        (?|?[ioe0-9])
                _e  "${3#?}"
                eval "exec $1&$?";;
        (?/*)   _e "$1"'"$2"' "${3#?}";;
        (*)     _e "$1"'"$2"' "$OLDPWD/${3#?}"
        esac    &&
        eval "  shift 3
                eval \" rm $$.$((_$$))
                        cd - >/dev/null
        \$@\"   $2$$.$((_$$)) &"
}

...so... I finally got around to making _p improved workable. The [-+] options can take any of ioe options to mean std(in|out|err) (though stdout is pretty useless in the command sub) or [0-9] to reference any descriptors you may have already setup, or anything else to be interpreted as any arbitrary filename. When called with + the argument is understood to be an out stream - as input is supposed to come from the command you call it with - and with - the argument is used as input.

And so, with this, the last sed above might be written like:

echo hi sed >"$(_pp +2 sed s/sed/mikeserv/)"

hi mikeserv

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.