Questions tagged [process-substitution]
Process substitution a form of inter-process communication that allows the input or output of a command to appear as a file (such as: `<(cmd)` or `>(cmd)`).
184
questions
0
votes
0
answers
38
views
docker build inline command fails with process substitution part
I am wondering why this docker build command fails:
docker build -t inline_img -f <(cat
`FROM node:12`
) .
[+] Building 0.1s (2/2) FINISHED
=> ERROR [internal] load build definition from 11 ...
- 9,718
0
votes
1
answer
88
views
Temporary symlink in shell - "named process substitution" - rename a file without creating a copy/symlink on the disk?
Thunderbird doesn't open files that don't have .eml extension as email files, but instead starts to compose a new message and adds them as attachments. To use thunderbird in scripts I'm looking for a ...
- 309
0
votes
0
answers
36
views
process substitution vs command substitution
I am trying to use vcf-merge command. The command works like this:
vcf-merge f1.vcf.gz f2.vcf.gz f3.vcf.gz
I have bunch of vcf.gz files so I did something like this:
vcf-merge <(ls *.vcf.gz | tr &...
- 2,491
0
votes
0
answers
81
views
Redirect output to /dev/fd/x in POSIX script, where /dev/fd/x replaces process substitution
I have a would-be POSIX script that includes filtering the stderr output:
exec <cmd> "$@" 2> >(grep -v "blih bluh blah")
Redirection does not occur because process ...
- 2,589
1
vote
2
answers
110
views
process substitution to mimic file
I can't recall how to accomplish this, I am trying to support users to be able to include plaintext in their commands for temp usage:
Here it reads from a file, file.json:
lp_solve --method='simplex' -...
- 9,718
2
votes
1
answer
155
views
Order of fd allocation in process substitution
Inspired by this answer https://security.stackexchange.com/a/166645
I am wondering the reason behind the weird ordering when I run these commands:
root@6cb8704148bf:/usr/app# echo <(printf "...
- 123
0
votes
1
answer
155
views
Named file descriptors with Process Substitution
I'm trying to use named file descriptors with Process Substitution.
I wrote the following code but it doesn't work:
# Open named file descriptors and associate to Process Substitution result
exec {...
- 2,495
0
votes
0
answers
18
views
process substitution for output: syntax error depending on how script is called [duplicate]
The following script results in error when called directly, but not when called with . ./test.sh:
% cat test.sh
echo "foo" |
tee >(rev)
% . ./process_subst_test.sh
foo
oof
% ./...
- 163
4
votes
1
answer
876
views
Why I need to type press Enter key to finish this command? [duplicate]
I'm a new Linux user and I was doing some experiments and trying to understand Process Substitution. I believe I already have a basic understanding of it. But here is a case that I don't know why. I'm ...
- 975
0
votes
0
answers
342
views
Using Process Substitution in docker as input <(command)
Recently, I faced some problems in developing bash pipelines with docker.
This regards to the use of process substitution [<()] as input for a specific docker command.
I know that this is a problem ...
0
votes
1
answer
99
views
Tesseract doesn't accept process substitution
I'm making a quick script that is supposed to use OCR tool (tesseract) on image in clipboard to convert it to text and output it. It looks like this:
#!/bin/sh
temp="$(mktemp tmpXXX.png)"
...
- 115
1
vote
0
answers
197
views
How to do shell variable expansion in process substitution? [closed]
I have a command like that in my script:
regex="abba"
diff <(grep "${regex}" file1) file2
The problem is if I run the script with bash script.sh the ${regex} variable doesn't ...
- 33
1
vote
1
answer
110
views
Bash script hangs when filtering stderr through sed
Introduction
I have a bash script to execute a command in multiple servers through ssh. It uses GNU parallel in the parallel version, a for loop in the sequential one.
The script is used like this:
...
1
vote
1
answer
347
views
bash script print stderr on failure
I am trying to make the bash script silent if everything goes fine but print out all the stderr and debug info if it crashes for some reason.
Below is what I have so far.
#!/usr/bin/bash
set -e
rm -...
- 4,367
0
votes
3
answers
479
views
exporting global variable in tee process substitution subshells [closed]
command | tee >(var="$(command1)") >(var1="$(command2)")
echo "$var $var1"
Also tried moreutils pee
command | pee 'var="$(command)"' 'var1="$(...
user414574
0
votes
2
answers
429
views
Bash process substitution: how to portably refer to standard input?
The question here is the bold sentence below. Suppose I'm trying to do this:
diff -u <(some command) {mystery-syntax}
I am diffing the output of some command and some data that comes from standard ...
- 7,904
0
votes
2
answers
2k
views
'-sh: syntax error: unexpected "("' when attempting process substitution on an embedded Linux device with `bash`
This uses "process substitution" (<()) and a "heredoc" (cat << EOF...EOF) to open a new bash process where it runs the startup "file" (--rcfile) containing alias ...
- 2,312
2
votes
1
answer
349
views
How does exec > work in AWS EC2 user data?
I was writing a Terraform module for AWS EC2 that involved executing a bash in the user data section. While I was developing I had an issue in the script I wrote but neither AWS nor Terraform provided ...
- 23
18
votes
4
answers
5k
views
CentOS 7: What is /bin/sh? It looks like Bash but seems to be something else
When I run a Centos 7 Docker Image like this
docker run -it centos:7 bash
Running something which is uses Process Substitution is fine (as expected, as Bash supports Process Substitution since the ...
- 1,962
0
votes
0
answers
51
views
What does this wecho command do?
I found a script that makes a file list. The list looks like this
/home/usr/example_directory
/home/usr/example_directory/file1.txt
/home/usr/example_directory/file2.txt
/home/usr/different_directory
/...
- 23
6
votes
2
answers
1k
views
What is more efficient or recommended for reading output of a command into variables in Bash?
If you want to read the single line output of a system command into Bash shell variables, you have at least two options, as in the examples below:
IFS=: read user x1 uid gid x2 home shell <<<...
- 590
1
vote
1
answer
193
views
Using named pipe to get process substitution exit codes
I have a script with a fairly complex command with several process substitutions and I need to get the exit codes from the commands in the process substitutions. I'm trying to do so via a named pipe ...
- 2,477
1
vote
2
answers
803
views
Pipe output from process substitution to variable
I have very complicated set of commands:
command | ... | ... | tee >(grep -c '[^3]$') >(grep -c '[^35]$') 1>/dev/null
I don't want to have a temporary file to save up output, as it is pretty ...
- 115
1
vote
1
answer
578
views
BusyBox tee is very slow
I am using Puppy-4.3.1 Linux on some older (Pentium 4) hardware.
I am getting terrible performance out of the tee command.
On this distro, tee is part of BusyBox v1.15.0.
For example, this command ...
- 215
2
votes
1
answer
165
views
Using same filename for the input in sub-shell and also as output in parent shell will conflict?
Referring to this answer of mine which I used the same tmp file for the input to the process-substitution and also writing the output to the same tmp file in parent shell, does this will cause any ...
- 41.1k
0
votes
1
answer
198
views
runit subprocess of service file not being killed on service stop
I was writing an runit service to restart picom on unlock based on this answer https://unix.stackexchange.com/a/439492/161514
#!/bin/bash
OBJECT_PATH=/org/freedesktop/login1/session/$XDG_SESSION_ID
...
1
vote
2
answers
261
views
Ordering output when two subshells write to stdout
I have a command of the below form
input | tee >(subshell) | mainshell
Both subshell and mainshell writes to stdout. So their output is not synchronised. E.g.
echo "Hello\nWorld" | tee &...
- 4,367
4
votes
1
answer
289
views
Process substitution with Perl script
I have a Perl script which requires the following arguments:
count.pl OUTPUT_FILE INPUT_FILE
What I want to do is to use process substitution instead of specifying INPUT_FILE explicitly, for instance:...
- 343
6
votes
1
answer
556
views
Difference between <(commands) and >(commands) in process substitution
In process substitution I can see two syntaxes:
>(command_list)
and
<(command_list)
Often I use the second syntax. But I have no idea what is exactly the difference between the two? searching ...
3
votes
2
answers
673
views
Zenity Cancel button for GNU parallel progress bar
As GNU parallel's manual shows, you can use a zenity progress bar with parallel:
seq 1000 | parallel -j30 --bar '(echo {};sleep 0.1)' \
2> >(zenity --progress --auto-kill) | wc
However, in ...
- 123
0
votes
1
answer
242
views
Can i use sudo in process substitution?
I want to calculate disk space via du, but I need sudo to access some dirs:
$perl -ane '$i+=($F[0] =~ s/^(\d+).*/$1/r); print "$i\n"' <(sudo du /home)
which gives:
[sudo] password for ...
- 381
2
votes
1
answer
465
views
1-liner process substitution for ksh
I have this command, which works fine for bash:
sh <(curl -sSl https://appi.sh/launch) (launches this interactive script)
I'd like to get this 1-liner working for ksh, too. Currently, it errors ...
- 2,137
2
votes
1
answer
448
views
Why is ssh-add <(echo 'pk contents') not working?
I am getting this error:
Permissions 0660 for '/dev/fd/63' are too open. It is required that
your private key files are NOT accessible by others. This private key
will be ignored.
when I run ...
user393961
1
vote
1
answer
711
views
how to keep stty sane after piping strace to vim?
this makes vim madness:
$strace -o >(vim -; stty sane) file.out; stty sane
I have typed stty sane in processed substitution as well as in next command but none of them did their job. Once i piped ...
- 320
6
votes
1
answer
696
views
Script not working when passed to bash via -c
Why does this work:
$ cat test.sh
#!/bin/bash
cat <(date|awk '{print $1}')
$ ./test.sh
Thu
but not as a command passed to bash:
$ bash -c "cat <(date|awk '{print $1}')"
Thu 2 Apr 2020 12:52:...
- 163
-1
votes
1
answer
195
views
Environment variables are not set when redirect echo through a pipeline at a function [closed]
This command:
echo '18' | source meta-cmf-raspberrypi/setup-environment
Not set the environment variables as instead happened with a simply:
source meta-cmf-raspberrypi/setup-environment
With "18" ...
2
votes
1
answer
66
views
Can't run file(1) on output of file descriptor, pipe, fifo, device, etc
I want to run the file(1) command on the output of another command, but all I get is
$ file -sL <(echo \#include \<stdio.h\>)
/dev/fd/63: ERROR: (null)
This works as expected in file-5.04 (...
- 592
-1
votes
4
answers
615
views
What is the difference between a heredoc and process substitution?
In this answer, it is explained how to use a heredoc to interact with interactive programs by connecting their stdin with a heredoc.
https://stackoverflow.com/questions/21130757/send-commands-to-...
- 297
0
votes
1
answer
983
views
How to pass a subprocess to a command with bash
I'm trying to pass multiple outputs to a command using sub-processes.
This is (a shortened version of) my command:
cat "$1"
one=cat "$1"
The output of command <(echo "foo") is
foo
/dev/fd/63: ...
- 3
0
votes
0
answers
200
views
Bash complex redirection (what does this command do)?
I recently was messing with an AWS virtual machine setup script. The documentation says to include this line:
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&...
- 101
0
votes
1
answer
116
views
Executing commands from process substitution in xinitrc
I felt that an .xinitrc like this
#!/bin/sh
bash <(sed -n 'H
/^exec ./{
x
s/\([^\n]\+ &\n\)*exec .*//
p
}' /etc/X11/xinit/xinitrc)
xmonad &
# ...
exec xterm
where ...
- 3,315
3
votes
3
answers
798
views
bash: race condition with pipeline, tee and process substitution
I want to see twice being output twice, but this script will only output it once:
dump() {
(sleep 1; cat) > "$1"
}
(sleep 0; echo "twice") | tee >(dump "./a.txt")
echo "$(< "a.txt")"
To ...
- 4,194
3
votes
1
answer
2k
views
paste: /dev/fd/63: No such file or directory
I'm new to Unix command. I was trying to execute the following commands in Git Bash and it directs to error message: paste: /dev/fd/63: No such file or directory
# Set Up Image Lists
paste <(awk "{...
- 31
1
vote
1
answer
117
views
Why does dd dislike taking input from process substitution?
Is there something about process substitution (which I think is implemented with unnamed pipes) that is incompatible with dd?
For example, this fails:
$ dd if=<(unzip -p raspbian.zip) of=/dev/sdb ...
- 359
6
votes
2
answers
8k
views
Run asynchronous tasks and retrieve their exit code and output in bash
I have to run a bunch of bash commands asynchronously and as soon as one finishes, I need to perform actions according to its exit code and output. Note that I can't predict how for long any of these ...
- 339
0
votes
0
answers
123
views
bash: how to pass variables to a loop within a process substitution
How do you pass variable values within a process substitution to a loop within that process substitution?
I'm reading a csv file into an associative array, and while reading each line, I would like ...
- 11
1
vote
1
answer
1k
views
zsh: inconsistent error handling in assignments
This is zsh -f:
Fereidoons-MacBook-Pro% local a=$(jaja) && echo bad
zsh: command not found: jaja
bad
Fereidoons-MacBook-Pro% a=$(jaja) && echo bad
zsh: command not found: jaja
...
- 1,543
1
vote
0
answers
69
views
How to use diff3 with text arguments instead of file paths?
I want to use diff3 with text arguments instead of file paths. For example, diff3 some_files/mine.txt some_files/older.txt ~/Downloads/yours.txt works as expected. However, I've been trying to achieve ...
- 111
4
votes
2
answers
373
views
Process Substitution in Bash
Can someone explain to me why I don't see the output of "date" from the below command? For N number for STDIN inputs, it's printing only for last (N-1) commands?
[root@RAJ-RHEL6 raj]# cat < <(...
- 43
0
votes
0
answers
129
views
Process substitution doesn't work with diff and node
I'm trying to compare the output of my fizzbuzz.js program:
for (let i = 1; i <= 100; i++) {
let out = "";
if (i % 3 == 0) out += "Fizz";
if (i % 5 == 0) out += "Buzz";
if (i % 3 &&...
- 455