Questions tagged [process-substitution]

Process substitution a form of inter-process communication that allows the input or output of a command to appear as a file (such as: `<(cmd)` or `>(cmd)`).

Filter by
Sorted by
Tagged with
3
votes
2answers
53 views

Process Substitution in Bash

Can someone explain to me why I don't see the output of "date" from the below command? For N number for STDIN inputs, it's printing only for last (N-1) commands? [root@RAJ-RHEL6 raj]# cat < <(...
80
votes
5answers
25k views

Process substitution and pipe

I was wondering how to understand the following: Piping the stdout of a command into the stdin of another is a powerful technique. But, what if you need to pipe the stdout of multiple commands? ...
0
votes
0answers
39 views

Process substitution doesn't work with diff and node

I'm trying to compare the output of my fizzbuzz.js program: for (let i = 1; i <= 100; i++) { let out = ""; if (i % 3 == 0) out += "Fizz"; if (i % 5 == 0) out += "Buzz"; if (i % 3 &&...
5
votes
0answers
43 views

cat in process substitution hangs: what is it waiting for? [duplicate]

I have this piece of code using Process Substitution. When I run it, it works fine, but it does not exit - just hangs there waiting for input. Why and how to overcome? prepro() { in=$(<...
11
votes
2answers
1k views

Why do I seem to lose data using this bash pipe construction?

I am trying to combine a few programs like so (please ignore any extra includes, this is heavy work-in-progress): pv -q -l -L 1 < input.csv | ./repeat <(nc "host" 1234) Where the source of ...
7
votes
3answers
123 views

Can I duplicate output of a pipe?

Well, the task is simple: a part of my script has to compute both md5 and sha1 hashes. The input is a file - big file - and hashes have to be put into MD and SH variables for later output composition. ...
0
votes
1answer
13 views

Process substitution to script in order to monitor iotop

I am actively trying to get the output of the iotop for monitoring. As you know, the iotop is interactive, meaning that once started, it'll provide live updates on disk usage on the system. I'm ...
0
votes
0answers
45 views

Write stdout to a temp file using command line

Say for example we wanted to write the env from the host to Docker container, one way would be to use a temp file: env > env_file && docker run --env-file env_file image_name to avoid a ...
1
vote
0answers
172 views

process substitution with <(gunzip …) fails

The following code returns: syntax error near ( #!/bin/bash myprogram -i <(gunzip -c file.gz) -j <(gunzip -c file2.gz) It seems it doesn't like brackets. How can I make this script work? If I ...
0
votes
1answer
50 views

How to send stdin to while loop using redirection operator

Not sure if I used the right words in the title, I am trying to do this: while read line; do echo "$line" exit; done; <( echo "foo\n"; exit ) but it hangs, I don't think the while loop ...
0
votes
0answers
46 views

How to pass to ssh the key stored in a shell environment variable?

Presumably ssh tries to access the fd several times, but the fd is destroyed after it has been read the very first time: # ssh -i <(echo $KEY) user@example.com Warning: Identity file /dev/fd/11 ...
13
votes
1answer
846 views

How to emulate Process Substitution in Dash?

In bash, I can use Process Substitution and treat output of a process as if it was a file saved on disk: $ echo <(ls) /dev/fd/63 $ ls -lAhF <(ls) lr-x------ 1 root root 64 Sep 17 12:55 /dev/fd/...
5
votes
1answer
154 views

Process substitution and redirection using tee

Suppose that you redirect, in bash, the standard output of a command cmd to a file named f.out, and the standard error to f.err, using tee to preserve console printing: cmd 1> >(tee f.out) 2>...
2
votes
2answers
412 views

How to detect an error using process substitution

This question is similar to the following link, but focused on using the command line (bash shell). Using a simple example, when doing the following command: $ cat <(date); echo $? Fri Jul 7 21:...
2
votes
1answer
81 views

concurrent processing in bash using process expansion, and redirection

Apart from possible races that have to be controlled by using proper synchronisation, it's possible in bash to feed a data source concurrently into multiple pipelines and collect all their outputs ...
0
votes
2answers
251 views

process substitution for opening a list of files with an application

I wish to use process substitution to direct a list of files (produced, for example, by ls or find) to a particular application for opening/viewing. While piping such a list to xargs is suitable for a ...
3
votes
1answer
471 views

Process substitution inside a subshell to set a variable

I'm trying to run a script remotely and use its standard output to populate a variable. I'm doing this to avoid temporary files. Here's the pattern I'm trying: var=$(bash <(curl -fsSkL http://...
2
votes
1answer
52 views

How to use bash process substitution for a sftp private key file?

With my team, we need to share a few private keys and we store them in a pass repository. So instead of extracting the key and storing it in cleartext on the filesystem, I am looking for a way to ...
18
votes
4answers
2k views

bash: how to propagate errors in process substitution?

I want my shell scripts to fail whenever a command executed with them fails. Typically I do that with: set -e set -o pipefail (typically I add set -u also) The thing is that none of the above ...
3
votes
4answers
242 views

Why does `… | sed 's/^/stdout: /'` print on empty stdin when `… > >(sed 's/^/stdout: /')` doesn't?

I'm trying to understand what causes the difference in these 2 constructs that I thought were functionally equivalent: $ ( echo foo >&2 ) 2> >(sed 's/^/stderr: /') | sed 's/^/stdout: /' ...
5
votes
1answer
164 views

Why is the stdout of a `>()` subshell different if it's part of a redirection (e.g. `> >()`)?

This is a followup question to: Why does `... | sed 's/^/stdout: /'` print on empty stdin when `... > >(sed 's/^/stdout: /')` doesn't? To be more concrete, why is this a ...
0
votes
1answer
63 views

Confused why script does not exit

I have this script verbatim: #!/usr/bin/env bash handle_json(){ while read line; do cat <<EOF {"@json-stdio":true,"value":{"mark":"$1","v":"$line"}} EOF done; } ( echo; echo; echo '...
3
votes
4answers
2k views

How to open process substituted file from php?

Here's what I tried to do myself: $ type 1.sh #!/bin/bash -eu php -r 'var_dump(file_get_contents($_SERVER["argv"][1]));' -- <(echo 1) $ ./1.sh PHP Warning: file_get_contents(/dev/fd/63): failed ...
0
votes
1answer
62 views

Is this bash argument substitution?

While studying bash process substitution, I found this: counter=0 while IFS= read -rN1 _; do ((counter++)) done < <(find /etc -printf ' ') echo "$counter files" If I understand correctly,...
-4
votes
2answers
80 views

Does process substitution need to be double quoted?

Double quoting command substitution is a good practice. Is it the same for process substitution (<() and >())? It seems double quotes allow command substitution, but disallow process ...
7
votes
4answers
1k views

Using process substitution to trick programs expecting files, with specific extensions as argument?

Here is my use case: the command line utility melt can accept a file name, with the extension .melt on the command line, and open it; as an example, this is a proper test_p.melt file: colour:blue out=...
1
vote
0answers
86 views

Using a here-doc with trailing backslashes inside a process substitution in bash

This: cat <<EOF one two three EOF Prints: one two three And this: cat <<EOF one \\ two \\ three EOF Prints: one \ two \ three But while this: cat <( cat <<EOF one two ...
4
votes
1answer
73 views

Specifying the file extension produced by zsh process substitution

I want to use zsh's process substitution to create a temporary file that can then be read by another program. However, the file it generates has no file extension, so the program reading it refuses to ...
3
votes
1answer
87 views

non-deterministic output from independent processes run concurrently by process substitution

On bash v4.1.2(2), the following simple statement, chosen merely as a minimal example demonstrating the problem, gives seemingly random output: $ for n in {0..1000}; do echo "$n"; done | tee >(...
3
votes
3answers
88 views

Pass input to multiple commands and compare their outputs

I am trying to pass standard input into multiple commands and compare their outputs. My current attempt seems close, but doesn't quite work - plus it relies on temporary files which I feel would not ...
5
votes
4answers
4k views

Why is there a “/dev/fd/63” in the output of “echo 123 >(cat)”?

$ echo 123 | cat 123 is doing what I expected, both commands run inside the same shell. But when I connect them with the >( ... ) expression, which connects the output of one command in the ...
2
votes
0answers
46 views

Command requires file, avoid creating temporary file [duplicate]

<() doesn't seem to do what I expect. This command requires a text file (it's JLinkExe, the text file contains commands for JLinkExe to run. But that shouldn't matter). JLinkExe -CommanderScript &...
0
votes
0answers
35 views

Process substitution of commands on separate lines of a string

I have a variable, COMMANDS, that contains a list of newline-separated commands (I know for example in this case that there will never be newlines, but they could be separated by NUL). For example, ...
18
votes
1answer
2k views

In zsh, difference between cat <(cat) vs cat | cat vs cat =(cat)?

I expected cat <(cat) and cat | cat to do the same thing: copy lines from stdin to stdout. My understanding was that both would execute a cat in a subshell, redirect the subshell cat's stdout to a ...
1
vote
1answer
39 views

Misunderstanding the purpose of a process substitution

I guess I'm missing some understanding on the use cases of a process substitution. My intuition was that a process substitution, of the form <(COMMANDS) would execute COMMANDS and then feed the ...
-1
votes
4answers
391 views

simple cat echo process substitution hangs

Simple process substitution with cat seems to hang: cat >( echo hello; ) never finishes. Also tried: cat >( echo hello; exit; ) Manually closing the standard out file descript from the ...
16
votes
2answers
874 views

The process substitution output is out of the order

The echo one; echo two > >(cat); echo three; command gives unexpected output. I read this: How process substitution is implemented in bash? and many other articles about process ...
0
votes
1answer
375 views

multiple pipes in loop, saving pipeline-result to array

I am trying to do the following (using bash): Search for files that always have the same name and extract data from these files. I want to store the extracted data in new arrays I am almost there, I ...
1
vote
2answers
58 views

How to create sh compatible bash function

I have this bash function: zmx(){ "$@" \ 2> >( while read line; do echo -e "r2g: $line"; done ) \ 1> >( while read line; do echo -e "r2g: $line"; done ) } this function is ...
10
votes
3answers
414 views

When was process substitution first introduced?

I'm curios to know how long process substitution has been around. What shell first had it? When did it get added to other shells?
0
votes
2answers
66 views

Which signal >(process) receives after main shell exits?

This is a Zshell question, although Bash, if it has >(command) syntax (i.e. process substitution of such kind), can hint a solution too. This really basic code explains all: % fun() { setopt ...
0
votes
1answer
202 views

Using process substitution, only send stderr to process

I have this: exec > >( while read line; do echo " stdout: $line"; done ) exec 2> >( while read line; do echo " stderr: $line"; done ) echo "rolo" >&2 echo "cholo" if you run that ...
26
votes
1answer
4k views

Why does BASH process substitution not work with some commands?

On occasion process substitution will not work as expected. Here is an example: Input: gcc <(echo 'int main(){return 0;}') Output: /dev/fd/63: file not recognized: Illegal seek collect2: error: ...
1
vote
1answer
355 views

Why process substitution is faster than pipe in this case? [duplicate]

I am wondering why ls -1 | while read file; do echo $file; tail -n 100 $file > >(sleep 1 && cat > $file) done is faster than ls -1 | while read file; do echo $...
0
votes
1answer
305 views

Process substitution vs FIFO (created by `makefifo`)

As far as I know, process substitution in bash and FIFO (created by mkfifo) are both named pipes. They both seem viable solutions for communication between processes. I was wondering how to decide ...
0
votes
1answer
52 views

When does a command in a process substitution or FIFO start and finish running?

When does a command in a process substitution in bash start and finish running? E.g. when does command1 in command2 >(command1) start and finish running respectively? when does command3 in ...
1
vote
2answers
1k views

cryptsetup: Attaching loopback device failed

I am using cryptsetup with detached header. For that I am using the --header parameter, to specify header file. I can open my device using: cryptsetup luksOpen /dev/sdb1 sdb1 --header LUKS-HEADER ...
1
vote
0answers
76 views

GPG error: “do_plaintext: wrote X bytes but expected Y bytes”

I'm having trouble using gpg when I use the <() syntax for process substitution. Here is a minimal-working example of my problem: $ gpg2 -c -o out.gpg <(head -c 73729 /dev/zero) gpg: ...
2
votes
3answers
85 views

Conditional process substitution

I have a command with a number of arguments: my_command --arg1 <arg1> \ --arg2 <arg2> \ --arg3 <arg3> I've wrapped the invocation of my_command in a function that sets up ...
3
votes
2answers
323 views

`tee` and `bash` process substitution order

I'd have supposed this code would have printed "oof" first: echo foo | tee >(rev) | ( sleep 1 ; cat ; ) Output: foo oof Increasing the sleep time doesn't change the order. Why doesn't that ...