Questions tagged [process-substitution]

Process substitution a form of inter-process communication that allows the input or output of a command to appear as a file (such as: `<(cmd)` or `>(cmd)`).

Filter by
Sorted by
Tagged with
80
votes
5answers
25k views

Process substitution and pipe

I was wondering how to understand the following: Piping the stdout of a command into the stdin of another is a powerful technique. But, what if you need to pipe the stdout of multiple commands? ...
37
votes
2answers
20k views

Why does process substitution result in a file called /dev/fd/63 which is a pipe?

I am trying to understand named pipes in the context of this particular example. I type <(ls -l) in my terminal and get the output as, bash: /dev/fd/63: Permission denied. If I type cat <(ls -...
33
votes
2answers
2k views

Why does `sort <(ls -l)` work but `sort < (ls -l)` fail?

Today I'm learning something about fifo with this article: Introduction to Named Pipes, which mentions cat <(ls -l). I did some experiments by using sort < (ls -l), which pops out an error: -...
26
votes
1answer
4k views

Why does BASH process substitution not work with some commands?

On occasion process substitution will not work as expected. Here is an example: Input: gcc <(echo 'int main(){return 0;}') Output: /dev/fd/63: file not recognized: Illegal seek collect2: error: ...
24
votes
1answer
2k views

What is the portable (POSIX) way to achieve process substitution?

Some shells, like bash, support Process Substitution which is a way to present process output as a file, like this: $ diff <(sort file1) <(sort file2) However, this construct isn't POSIX and, ...
18
votes
1answer
2k views

In zsh, difference between cat <(cat) vs cat | cat vs cat =(cat)?

I expected cat <(cat) and cat | cat to do the same thing: copy lines from stdin to stdout. My understanding was that both would execute a cat in a subshell, redirect the subshell cat's stdout to a ...
18
votes
4answers
2k views

bash: how to propagate errors in process substitution?

I want my shell scripts to fail whenever a command executed with them fails. Typically I do that with: set -e set -o pipefail (typically I add set -u also) The thing is that none of the above ...
16
votes
2answers
868 views

The process substitution output is out of the order

The echo one; echo two > >(cat); echo three; command gives unexpected output. I read this: How process substitution is implemented in bash? and many other articles about process ...
13
votes
2answers
956 views

Bash: process substitution and stdin

The following line is obvious: echo "bla" | foo | bar But does the ones below do the same? echo "bla" | bar <(foo) echo "bla" | bar < <(foo) Which of the foo and bar read "bla" from stdin ...
13
votes
4answers
2k views

How do I capture the exit code / handle errors correctly when using process substitution?

I have a script that parses file names into an array using the following method taken from a Q&A on SO: unset ARGS ARGID="1" while IFS= read -r -d $'\0' FILE; do ARGS[ARGID++]="$FILE" done &...
13
votes
1answer
844 views

How to emulate Process Substitution in Dash?

In bash, I can use Process Substitution and treat output of a process as if it was a file saved on disk: $ echo <(ls) /dev/fd/63 $ ls -lAhF <(ls) lr-x------ 1 root root 64 Sep 17 12:55 /dev/fd/...
13
votes
3answers
3k views

How to combine Bash's process substitution with HERE-document?

In Bash version 4.2.47(1)-release when I try to catenate formatted text that comes from a HERE-dcoument like so: cat <(fmt --width=10 <<FOOBAR (I want the surrounding parentheses to be part ...
12
votes
2answers
2k views

How process substitution is implemented in bash?

I was researching the other question, when I realized I don't understand what's happening under the hood, what are those /dev/fd/* files and how come child processes can open them.
11
votes
5answers
39k views

grep files from list

I am trying to run grep against a list of a few hundred files: $ head -n 3 <(cat files.txt) admin.php ajax/accept.php ajax/add_note.php However, even though I am grepping for a string that I know ...
11
votes
3answers
12k views

Failed to open config file '/dev/fd/63', error: No such file or directory for wpa_supplicant

When I do this: sudo wpa_supplicant -D nl80211,wext -i wlp4s0 -c <(wpa_passphrase "some ssid" "password") I get Successfully initialized wpa_supplicant Failed to open config file '/dev/fd/63', ...
11
votes
2answers
1k views

Why do I seem to lose data using this bash pipe construction?

I am trying to combine a few programs like so (please ignore any extra includes, this is heavy work-in-progress): pv -q -l -L 1 < input.csv | ./repeat <(nc "host" 1234) Where the source of ...
11
votes
3answers
13k views

How do I use tee to redirect to grep

I don't have much experience of using tee, so I hope this is not very basic. After viewing one of the answers to this question I came across a strange beheviour with tee. In order for me to output ...
11
votes
1answer
2k views

Why doesn't the process substitution <() work with ssh -F

I have some vagrant virtual machines. To log into them I issue the vagrant ssh command. I want to log into them using regular ssh command. The vagrant ssh-config outputs the suitable config file $ ...
11
votes
2answers
418 views

Output order with process substitution

This is what I usually do to run grep and wc on a file without having to scan it twice <file.txt tee >(grep LITERAL) >(wc -l) >/dev/null However, this yields EXEC LITERAL 32 sometimes ...
10
votes
3answers
414 views

When was process substitution first introduced?

I'm curios to know how long process substitution has been around. What shell first had it? When did it get added to other shells?
9
votes
1answer
1k views

Process substitution in GNU Makefiles

On a bash prompt, one can execute diff using pseudo files: diff <(echo test) <(echo test) Adding this as is into a Makefile fails: all: diff <(echo test) <(echo test) The error ...
9
votes
2answers
428 views

How bash treats “> >()”

While experimenting with output redirection and process substitution I stumbled upon the following command and its resulting output: me@elem:~$ echo foo > >(cat); echo bar bar me@elem:~$ ...
9
votes
3answers
2k views

Dash equivalent of self-redirection of script output

In Bash you can redirect all future stdout output of the currently running script. For example with this script, exec > >(logger -t my-awesome-script) echo 1 echo 2 echo 3 This will end up in ...
8
votes
4answers
1k views

Split an input for different command and combine the result

I know how to combine the result of different command paste -t',' <(commanda) <(commandb) I know pipe same input to different command cat myfile | tee >(commanda) >(commandb) Now how ...
8
votes
1answer
1k views

bash wait for process in process substitution even if command is invalid [duplicate]

I'm trying to write a utility script errpipe with a simple api that runs stderr through a filter. At first I tried to implement it using bash's process substitution feature. #!/bin/bash com="$1" ...
8
votes
1answer
373 views

What are the difference between those four commands (fifo, process substitution, redirection…)

My goal is to create a simple echo server using nc and a single fifo. I'm not looking for the best way to do it, I'm merely trying to understand the semantics of the following commands (when does fork ...
8
votes
1answer
693 views

bash process substitution with temporary file

Some programs needs their files to be seekable, for example objdump does. $ objdump -D -b binary -m i8086 <(echo 0xea 0x5b 0xe0 0x00 0xf0|xxd -r -p) objdump: Warning: '/proc/self/fd/11' is not an ...
7
votes
2answers
2k views

Process substitution with input redirection

I am trying to understand input redirection in combination with process substituation. I am using bash 3 An example with tr is the following $ tr "o" "a" <(echo "Foo") tr: extra operand `/dev/fd/...
7
votes
3answers
123 views

Can I duplicate output of a pipe?

Well, the task is simple: a part of my script has to compute both md5 and sha1 hashes. The input is a file - big file - and hashes have to be put into MD and SH variables for later output composition. ...
7
votes
1answer
858 views

Wait for Bash subshells

I pipe the same content to multiple commands with tee, subshells and redirects like this: #!/usr/bin/env bash echo 'hello' | tee \ >( sleep 3; cat /dev/stdin ) \ >( sleep 2; cat /dev/stdin ...
7
votes
4answers
1k views

Using process substitution to trick programs expecting files, with specific extensions as argument?

Here is my use case: the command line utility melt can accept a file name, with the extension .melt on the command line, and open it; as an example, this is a proper test_p.melt file: colour:blue out=...
7
votes
1answer
240 views

Why does OS X `install` errors on redirected input when the same version of `install` on linux works fine?

I would like to use install command in order to create a new executable file with pre-populated content (e.g. with single pwd command in it). So I've extended this example which creates a new empty ...
6
votes
2answers
11k views

Running remote command with ssh getting “bash: /dev/fd/63: No such file or directory”

I have /root/test.sh on a remote server, which has the following content: #!/bin/bash date On the remote server, I get the following output: # ./test.sh Fri Dec 18 07:41:10 EST 2015 # bash <(...
6
votes
4answers
2k views

Bash Reuse Process Substitution File

I have a big script which takes a file as input and does various stuff with it. Here is a test version: echo "cat: $1" cat $1 echo "grep: $1" grep hello $1 echo "sed: $1" sed 's/hello/world/g' $1 I ...
6
votes
2answers
3k views

Process substitution with tee and paste

I'm trying to pipe a command through the output of two other commands and then merge the results of the two process substitutions. An example that gets me close is: command | tee >(sed -rn 's/.*...
6
votes
1answer
2k views

Performance differences between pipelines and process substitution

I tend to use pipelines in my bash scripts over process substitution in most situations, especially in cases of using multiple sets of commands as it seems more readable to do ... | ... | ... over ... ...
6
votes
2answers
600 views

Combining multiple process substitution

Suppose you tried something like this: $ paste ../data/file-{A,B,C}.dat and realize that you want to sort each file (numerically, let's suppose) before pasting. Then, using process substitution, you ...
5
votes
4answers
4k views

Why is there a “/dev/fd/63” in the output of “echo 123 >(cat)”?

$ echo 123 | cat 123 is doing what I expected, both commands run inside the same shell. But when I connect them with the >( ... ) expression, which connects the output of one command in the ...
5
votes
1answer
1k views

Pipes vs process substitution

While trying the cat "$@" trick to read from either standard input or given files, it turned out that pipe and process substitution handle a missing trailing newline differently: printf %s foo > ...
5
votes
2answers
415 views

Understanding i/o redirection in the context of _process substitution_

Running GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu). I don't really understand process substitution (ProcSub) from the perspective of a user interested in lifting the hood on i/o ...
5
votes
2answers
5k views

Process substitution from curl to bash as root

I'm trying to run a script from URL as root with this command: sudo bash <(curl -s http://copy.com/gLVZIqUubzcS/popcorn) But I'm getting this error: bash: /dev/fd/63: Aucun fichier ou dossier de ...
5
votes
1answer
154 views

Process substitution and redirection using tee

Suppose that you redirect, in bash, the standard output of a command cmd to a file named f.out, and the standard error to f.err, using tee to preserve console printing: cmd 1> >(tee f.out) 2>...
5
votes
1answer
164 views

Why is the stdout of a `>()` subshell different if it's part of a redirection (e.g. `> >()`)?

This is a followup question to: Why does `... | sed 's/^/stdout: /'` print on empty stdin when `... > >(sed 's/^/stdout: /')` doesn't? To be more concrete, why is this a ...
5
votes
0answers
43 views

cat in process substitution hangs: what is it waiting for? [duplicate]

I have this piece of code using Process Substitution. When I run it, it works fine, but it does not exit - just hangs there waiting for input. Why and how to overcome? prepro() { in=$(<...
4
votes
2answers
874 views

Putting process substituted file descriptor into variable

I have long command roughly this: $ command-outer "long command which references a file here: filename1, and another file reference here: filename2" The files are the output of another command. So I ...
4
votes
1answer
455 views

Bash - process substitution in chroot without /dev and /proc

I have a chroot jail without /dev and /proc mounted. When I try to do process substitution here, this happens: bash-4.4# cat <(ls) cat: open /dev/fd/63: No such file or directory I tried to ...
4
votes
1answer
72 views

Specifying the file extension produced by zsh process substitution

I want to use zsh's process substitution to create a temporary file that can then be read by another program. However, the file it generates has no file extension, so the program reading it refuses to ...
4
votes
3answers
13k views

syntax error near unexpected token `<'

I am writing a script to read the output of a command to variable a and b. This is the script #!/bin/bash read a b < <(awk '/Application Server/ && !seen[$7]++{printf "%s ", $7}' /tmp/...
4
votes
1answer
542 views

Diff the output of the last two commands

I frequently find myself doing something like the following: $ big long command 1 $ big long command 2 $ diff -u <(big long command 1) <(big long command 2) How can I get the shell (bash in my ...
4
votes
1answer
537 views

File descriptors and redirect in bash

Can someone explain why this command works: gpg --enable-special-filenames --verify --batch \ <(curl -s https://www.gnupg.org/ftp/gcrypt/gnupg/gnupg-2.0.29.tar.bz2.sig) \ <(curl https://www....