Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [process-substitution]

Process substitution a form of inter-process communication that allows the input or output of a command to appear as a file (such as: `<(cmd)` or `>(cmd)`).

5
votes
1answer
109 views

Process substitution and redirection using tee

Suppose that you redirect, in bash, the standard output of a command cmd to a file named f.out, and the standard error to f.err, using tee to preserve console printing: cmd 1> >(tee f.out) 2>...
2
votes
1answer
61 views

concurrent processing in bash using process expansion, and redirection

Apart from possible races that have to be controlled by using proper synchronisation, it's possible in bash to feed a data source concurrently into multiple pipelines and collect all their outputs ...
3
votes
1answer
262 views

Process substitution inside a subshell to set a variable

I'm trying to run a script remotely and use its standard output to populate a variable. I'm doing this to avoid temporary files. Here's the pattern I'm trying: var=$(bash <(curl -fsSkL http://...
2
votes
1answer
37 views

How to use bash process substitution for a sftp private key file?

With my team, we need to share a few private keys and we store them in a pass repository. So instead of extracting the key and storing it in cleartext on the filesystem, I am looking for a way to ...
5
votes
1answer
101 views

Why is the stdout of a `>()` subshell different if it's part of a redirection (e.g. `> >()`)?

This is a followup question to: Why does `... | sed 's/^/stdout: /'` print on empty stdin when `... > >(sed 's/^/stdout: /')` doesn't? To be more concrete, why is this a ...
3
votes
4answers
145 views

Why does `… | sed 's/^/stdout: /'` print on empty stdin when `… > >(sed 's/^/stdout: /')` doesn't?

I'm trying to understand what causes the difference in these 2 constructs that I thought were functionally equivalent: $ ( echo foo >&2 ) 2> >(sed 's/^/stderr: /') | sed 's/^/stdout: /' ...
0
votes
1answer
61 views

Confused why script does not exit

I have this script verbatim: #!/usr/bin/env bash handle_json(){ while read line; do cat <<EOF {"@json-stdio":true,"value":{"mark":"$1","v":"$line"}} EOF done; } ( echo; echo; echo '...
0
votes
1answer
62 views

Is this bash argument substitution?

While studying bash process substitution, I found this: counter=0 while IFS= read -rN1 _; do ((counter++)) done < <(find /etc -printf ' ') echo "$counter files" If I understand correctly,...
-5
votes
2answers
71 views

Does process substitution need to be double quoted?

Double quoting command substitution is a good practice. Is it the same for process substitution (<() and >())? It seems double quotes allow command substitution, but disallow process ...
1
vote
0answers
54 views

Using a here-doc with trailing backslashes inside a process substitution in bash

This: cat <<EOF one two three EOF Prints: one two three And this: cat <<EOF one \\ two \\ three EOF Prints: one \ two \ three But while this: cat <( cat <<EOF one two ...
4
votes
1answer
63 views

Specifying the file extension produced by zsh process substitution

I want to use zsh's process substitution to create a temporary file that can then be read by another program. However, the file it generates has no file extension, so the program reading it refuses to ...
3
votes
3answers
70 views

Pass input to multiple commands and compare their outputs

I am trying to pass standard input into multiple commands and compare their outputs. My current attempt seems close, but doesn't quite work - plus it relies on temporary files which I feel would not ...
2
votes
0answers
44 views

Command requires file, avoid creating temporary file [duplicate]

<() doesn't seem to do what I expect. This command requires a text file (it's JLinkExe, the text file contains commands for JLinkExe to run. But that shouldn't matter). JLinkExe -CommanderScript &...
0
votes
0answers
33 views

Process substitution of commands on separate lines of a string

I have a variable, COMMANDS, that contains a list of newline-separated commands (I know for example in this case that there will never be newlines, but they could be separated by NUL). For example, ...
-1
votes
4answers
280 views

simple cat echo process substitution hangs

Simple process substitution with cat seems to hang: cat >( echo hello; ) never finishes. Also tried: cat >( echo hello; exit; ) Manually closing the standard out file descript from the ...
0
votes
1answer
263 views

multiple pipes in loop, saving pipeline-result to array

I am trying to do the following (using bash): Search for files that always have the same name and extract data from these files. I want to store the extracted data in new arrays I am almost there, I ...
1
vote
2answers
57 views

How to create sh compatible bash function

I have this bash function: zmx(){ "$@" \ 2> >( while read line; do echo -e "r2g: $line"; done ) \ 1> >( while read line; do echo -e "r2g: $line"; done ) } this function is ...
0
votes
2answers
58 views

Which signal >(process) receives after main shell exits?

This is a Zshell question, although Bash, if it has >(command) syntax (i.e. process substitution of such kind), can hint a solution too. This really basic code explains all: % fun() { setopt ...
0
votes
1answer
160 views

Using process substitution, only send stderr to process

I have this: exec > >( while read line; do echo " stdout: $line"; done ) exec 2> >( while read line; do echo " stderr: $line"; done ) echo "rolo" >&2 echo "cholo" if you run that ...
1
vote
1answer
302 views

Why process substitution is faster than pipe in this case? [duplicate]

I am wondering why ls -1 | while read file; do echo $file; tail -n 100 $file > >(sleep 1 && cat > $file) done is faster than ls -1 | while read file; do echo $...
0
votes
1answer
249 views

Process substitution vs FIFO (created by `makefifo`)

As far as I know, process substitution in bash and FIFO (created by mkfifo) are both named pipes. They both seem viable solutions for communication between processes. I was wondering how to decide ...
0
votes
1answer
48 views

When does a command in a process substitution or FIFO start and finish running?

When does a command in a process substitution in bash start and finish running? E.g. when does command1 in command2 >(command1) start and finish running respectively? when does command3 in ...
1
vote
0answers
67 views

GPG error: “do_plaintext: wrote X bytes but expected Y bytes”

I'm having trouble using gpg when I use the <() syntax for process substitution. Here is a minimal-working example of my problem: $ gpg2 -c -o out.gpg <(head -c 73729 /dev/zero) gpg: ...
2
votes
3answers
77 views

Conditional process substitution

I have a command with a number of arguments: my_command --arg1 <arg1> \ --arg2 <arg2> \ --arg3 <arg3> I've wrapped the invocation of my_command in a function that sets up ...
1
vote
1answer
36 views

Misunderstanding the purpose of a process substitution

I guess I'm missing some understanding on the use cases of a process substitution. My intuition was that a process substitution, of the form <(COMMANDS) would execute COMMANDS and then feed the ...
3
votes
1answer
74 views

non-deterministic output from independent processes run concurrently by process substitution

On bash v4.1.2(2), the following simple statement, chosen merely as a minimal example demonstrating the problem, gives seemingly random output: $ for n in {0..1000}; do echo "$n"; done | tee >(...
0
votes
0answers
93 views

Bash Process Substitution Hangs When Listing Bindings

TLDR: cat <(INPUTRC=/dev/null bash -c 'bind -pm emacs') # freezes # I can't use this because I need to pipe the output of the bind cat <(INPUTRC=/dev/null bash -c 'bind -pm emacs' &) # ...
1
vote
1answer
328 views

Iteratively write to the same process substitution file on MacOs

The following bash-fu code works fine on Linux but breaks on MacOS: files="foo bar" echo PROG 1 for file in $files do echo $file | tee -a tempfile.txt done sort -u tempfile.txt echo PROG 2 ...
3
votes
2answers
274 views

`tee` and `bash` process substitution order

I'd have supposed this code would have printed "oof" first: echo foo | tee >(rev) | ( sleep 1 ; cat ; ) Output: foo oof Increasing the sleep time doesn't change the order. Why doesn't that ...
4
votes
1answer
402 views

Bash - process substitution in chroot without /dev and /proc

I have a chroot jail without /dev and /proc mounted. When I try to do process substitution here, this happens: bash-4.4# cat <(ls) cat: open /dev/fd/63: No such file or directory I tried to ...
13
votes
2answers
761 views

The process substitution output is out of the order

The echo one; echo two > >(cat); echo three; command gives unexpected output. I read this: How process substitution is implemented in bash? and many other articles about process ...
4
votes
1answer
601 views

Difference between subshells and process substitution

In bash, I want to assign my current working directory to a variable. Using a subshell, I can do this. var=$(pwd) echo $var /home/user.name If I use process substitution like so: var=<(pwd) ...
8
votes
1answer
997 views

bash wait for process in process substitution even if command is invalid [duplicate]

I'm trying to write a utility script errpipe with a simple api that runs stderr through a filter. At first I tried to implement it using bash's process substitution feature. #!/bin/bash com="$1" ...
0
votes
1answer
299 views

Difference between 'cmd1 | cmd2' and 'cmd2 <(cmd1)' and 'mkfifo foo; cmd1>foo; cmd2 foo'?

Or: When and why to not prefer STDIN when cmd2 can use both, and is a single command? I come across questions on this site, where 3 of their answers are one of each of the above, like this one. This ...
1
vote
2answers
273 views

bash timeout command kills randomly before duration

I'm trying to show thumbnails from youtube for specific timeout to user and then show next one I made this command: while read line; do\ wget -O- -q "$line" | \ timeout 10s display ;\ ...
0
votes
1answer
56 views

Equivalence to the usage of process substitution?

Is the usage of process substitution command1 <(command2) equivalent to command2 > ~/myfile command1 ~myfile ? Can the two above be replaced with each other? Does command1 and command2 ...
3
votes
2answers
203 views

Sensitive data in process substitution

Let's say I do: #!/bin/bash #content=$(cat -) content="foo" pass=$1 echo $content | ccrypt -f -k <(echo -n $pass) Can the permission set on the process substition /dev/fd/* files be trusted to ...
1
vote
2answers
248 views

How to execute an output string without bash's command substitution?

How to execute an output string without bash's command substitution? For example, if I type in terminal tshark, and the program isn't installed yet, I will recieve an error message like: _some text_ ...
2
votes
2answers
346 views

How to detect an error using process substitution

This question is similar to the following link, but focused on using the command line (bash shell). Using a simple example, when doing the following command: $ cat <(date); echo $? Fri Jul 7 21:...
5
votes
4answers
3k views

Why is there a “/dev/fd/63” in the output of “echo 123 >(cat)”?

$ echo 123 | cat 123 is doing what I expected, both commands run inside the same shell. But when I connect them with the >( ... ) expression, which connects the output of one command in the ...
0
votes
1answer
607 views

Subshell and process substitution

Apologies if this is a basic question - I'm stuck trying to solve a larger problem, and it's come down to how a shell script is invoked - directly (shellScript.sh) or using sh shellScript.sh. Here's ...
9
votes
1answer
1k views

Process substitution in GNU Makefiles

On a bash prompt, one can execute diff using pseudo files: diff <(echo test) <(echo test) Adding this as is into a Makefile fails: all: diff <(echo test) <(echo test) The error ...
6
votes
1answer
760 views

Wait for Bash subshells

I pipe the same content to multiple commands with tee, subshells and redirects like this: #!/usr/bin/env bash echo 'hello' | tee \ >( sleep 3; cat /dev/stdin ) \ >( sleep 2; cat /dev/stdin ...
3
votes
1answer
234 views

Join two unsorted files with POSIX? [duplicate]

Because the input to join must be sorted, often the command is called similarly to: join <(sort file1) <(sort file2) This is not portable as it uses process substitution, which is not ...
1
vote
0answers
150 views

process substitution with <(gunzip …) fails

The following code returns: syntax error near ( #!/bin/bash myprogram -i <(gunzip -c file.gz) -j <(gunzip -c file2.gz) It seems it doesn't like brackets. How can I make this script work? If I ...
4
votes
1answer
115 views

Change location of zsh's process substitution ‘=()’

zsh is apparently using /tmp/zshXXXXXX to store temporary files created by the process substitution using =(). For example, $ echo =(echo test) /tmp/zsh4RmpQZ However, I would like to change the ...
0
votes
1answer
100 views

Process Substitution (e. g. $(somecommand) ) In A Nutshell

On my shell from home directory on mac, I run the following eval "ls" and this just shows me the directories. But if I run eval "$(ls)" I get the following output: zsh: command not found: ...
1
vote
1answer
388 views

Is the command in a process substitution invoked in a subshell?

From the bash manual Process substitution is supported on systems that support named pipes (fifos) or the /dev/fd method of naming open files. It takes the form of <(list) or >...
3
votes
1answer
149 views

Why ssh immediately disconnects when using -F /dev/stdin syntax?

I've configured and booted default Ubuntu VM box by the following commands: vagrant init ubuntu/trusty64; vagrant up --provider virtualbox Now, for education purposes, I'd like to use the following ...
1
vote
1answer
564 views

Piping multiple inputs into Ghostscript

I am trying to create a one-line Linux command to combine two PDF files - which are downloaded from a URL - using Ghostscript. However, I do not want to create any temporary files (everything should ...