Skip to main content

Questions tagged [pipe]

A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in <unistd.h>. Shells provide pipe creation between processes using "|".

Filter by
Sorted by
Tagged with
2 votes
1 answer
50 views

Is there a way to `exec` a pipeline or exit shell after launching a pipeline?

I'm writing a shell wrapper script that is supposed to act as a pager (receiving input on stdin and performing output on a tty). This wrapper prepares environment and command lines, then launches two ...
intelfx's user avatar
  • 5,466
0 votes
1 answer
44 views

how to pipe output of find into eog

I'm trying to view photos in a slideshow which are nested within other folders but can be found with the find command. The find command on its own does find all the listed photos but this - find /...
cardamom's user avatar
  • 602
0 votes
2 answers
113 views

Terminating the whole pipeline at once when its last process ends

Given the pipeline printf '%s\n' 1 2 3 4 5 6 7 8 9 | while read -r num do echo "$num" > /dev/stderr echo "$num" done | while read -r num do echo $(( $num * 10 )) [ &...
fuumind's user avatar
  • 449
0 votes
1 answer
53 views

Piped dd command limits its output to 65536 bytes, why?

My colleague and I are both using a Docker container running Ubuntu 22.04.4 LTS (Jammy Jellyfish): root@c2155d7af4a4:/# grep VERSION= /etc/os-release VERSION="22.04.4 LTS (Jammy Jellyfish)" ...
Bart Vandewoestyne's user avatar
1 vote
2 answers
82 views

Is it possible to read from stderr in a bash script?

I have a bash script like this: #!/bin/bash while read -r -a line do ... parse $line in some way done This script is executed by piping the command from another program: some-random-program | ...
lord_nimon's user avatar
4 votes
1 answer
280 views

pipe to a different file descriptor

Following up on Using while loop to ssh to multiple servers, i.e., for while IFS= read -r -u9 HOST ; do ssh "$HOST" "uname -a" ; done 9< servers.txt which reads from a ...
xpt's user avatar
  • 1,644
0 votes
1 answer
47 views

Use grep to search stdout, without replacing stdout

At the moment, I have this in a wrapper script: 2>&1 ./update.sh | ts | tee -a ./update.log and this in update.sh: if apt full-upgrade -y | grep linux-headers then echo echo Need to ...
AaronD's user avatar
  • 153
3 votes
1 answer
283 views

SIGPIPE and bash pipefail

I am trying to better understand SIGPIPE on linux. I ran this experiment: { ls -al /tmp/ ; echo "$?" 1>&2 ; } | head and it echos 141 which I understand is an exit code that is given ...
Dave Butler's user avatar
0 votes
1 answer
36 views

What is a good way to perform a SHA256d hash (double SHA256) on an OpenBSD fresh install?

What is a good way to perform a SHA256d hash (double SHA256) on the default terminal of an network isolated OpenBSD fresh install? Here's what I'm doing: echo test > testfile cat testfile | openssl ...
Lee's user avatar
  • 519
0 votes
0 answers
118 views

How to keep a shell alive after it gets started from inside a program?

I was not sure where to ask this question, since it is a mix of both programming and Linux, but hopefully this is the right place. I have written a simple program, which I'm buffer overflowing. I have ...
r3k0j's user avatar
  • 15
0 votes
1 answer
45 views

Archiving stdout to multiple tapes

I have large files which are generated on the fly to stdout, one every 24hours. I would like to archive these files progressively on tapes, ideally in a single archive which potentially spans multiple ...
lorenzo's user avatar
0 votes
1 answer
17 views

rsyslogd v3.x.x unexpectedly closes write connection to named pipe target

I've configured rsyslog to forward certain log messages to a named pipe /tmp/logger.pipe. I then have a separate process reading from the named pipe. Relevant section from /etc/rsyslog.conf # Remote ...
Izzo's user avatar
  • 971
0 votes
1 answer
82 views

Performing character-level comparison of "strings" without explicitly creating temporary files for git diff

Referring to this https://stackoverflow.com/a/31356602, I wrote this code: #!/bin/bash # Define the two strings to compare string1="First string with some random text." string2="Second ...
Gabriele's user avatar
  • 333
0 votes
1 answer
56 views

find with execdir, pipe redirection evaluated too early (quote issue?)

I use multiple GitHub accounts at work, so I need to modify the remote url to add my work account. I have figured out the recursive find -execdir, and how to get/sed/set the remote url, but I cannot ...
Robert Altman's user avatar
0 votes
1 answer
31 views

Intercepting/debugging wrapper over a program with bidirectional command/response interface

I have a command-line program, the target, that implements a command/response interface over stdio (for other programs, not for humans). Specifically: the consumer of this program starts it up reads ...
intelfx's user avatar
  • 5,466
0 votes
0 answers
41 views

What does rsyslog do if it fills up a named pipe?

I'm working on a C application that is intended to monitor and process the output of rsyslog. My research shows that a way to do this is to create a named pipe and then create a new syslog target that ...
Izzo's user avatar
  • 971
0 votes
0 answers
36 views

Reading from FIFO in nc (netcat)

I have a bunch of log files that I'd like to watch as they get updated. I came up with the script below: #!/bin/bash # Check if at least one log file is provided if [ "$#" -eq 0 ]; then ...
farhanhubble's user avatar
0 votes
0 answers
124 views

Understanding bash read variable substitution [duplicate]

First of all, I have little experience using Bash and I apologize for my bad English. Maybe it's obvious. I am trying to understand why Bash drop value of the variable in this oneliner. echo "...
bas1c's user avatar
  • 1
0 votes
1 answer
70 views

Why is tee stopping in the middle when piped with nohup?

I'm running a script (in a Laravel project) that takes a few hours to complete. nohup to prevent it from exiting if my ssh session is disconnected. nohup php artisan do:thing 2>&1 | tee ~/tmp ...
Jeremy Belolo's user avatar
0 votes
0 answers
16 views

How to preserve updating output in pipe, e.g. when output has `\r`

Whenever I pipe some output that is updating (say with \r), e.g. to a pager (less), or to colourise parts of it, the pipe seems to drop the lines that are updating. Any idea how to preserve the ...
suvayu's user avatar
  • 150
2 votes
1 answer
280 views

Kill process on broken pipe from within a bash script

Consider the code: printf '%s\n' 1 2 3 4 5 | head -n 2 which has the output: 1 2 My understanding is that when the head process breaks the pipe after reading the two first lines then the printf ...
fuumind's user avatar
  • 449
0 votes
1 answer
78 views

SSH's -t option - how do I remotely execute a complex command that includes pipes and variable expansion?

The following script is run by root and is meant to share two directories from jim's computer to that of oumaima. #!/usr/bin/bash au=$(echo "$(head -n 1 /etc/doas.conf)"|sed 's/permit//;s/...
John Smith's user avatar
0 votes
0 answers
85 views

What is the correct way to pipe the same stream to multiple commands

I have used this simple form with partial success: $ command1 | (command2; command3; command4...) But this fails in some cases. For example: $ seq 100 | (head -10; echo "done") # works fine $...
user167612's user avatar
  • 1,700
0 votes
0 answers
31 views

Skip pipe when empty in bash [duplicate]

How can I let the pipe work only when there's output in first command? I'm looking for a workaround for pick. I've tried find|xargs -r pick, but pick only receives input from pipe. Thanks in advance!
Rashedul Hasan Rafi's user avatar
0 votes
1 answer
67 views

Capture output from SOX "-n stat"

I am trying to capture/pipe the output from the following: arecord -f S16_LE -qd 5 file && sox file -n stat output: Samples read: 8000 Length (seconds): 1.000000 Scaled by: ...
Mark Miller's user avatar
0 votes
2 answers
323 views

Extract parts of a string using head and tail only

Hello I would like to know if there is a way where I can only use head, tail, and pipes (and redirection eventually) to extract and output the start, middle characters, and end of a string Example: ...
naynay's user avatar
  • 17
1 vote
2 answers
367 views

Head/Tail command to grab multiple sets of lines

I have to grab the first two lines, the lines 43 and 44, and the last 2 lines from a file in one conduct of commands. Is there away to print those while only using head, tail and pipe commands AND ...
cow's user avatar
  • 81
1 vote
0 answers
43 views

Why can't I cat a file, pipe it to jq, and redirect the output to another file? [duplicate]

I want to use jq to pretty-print a JSON-format file, and pipe the result to another json file. My first instinct was cat foo.json | jq > bar.json but this results in an empty bar.json: $ cat <&...
StoneThrow's user avatar
  • 1,797
0 votes
1 answer
69 views

Is there a command that makes another command "in-place"?

I find myself trying to "edit" a file in-place quite a lot in my shell. Say, I want to remove all lines in words.txt that contain the letter e. Then, I would execute: $ grep -v e words.txt &...
Lorenz Leutgeb's user avatar
0 votes
0 answers
50 views

Circular pipe situation

More of an academic/theory question - say I have process A piping into process B: A | B normally, the way pipes are designed, is if process A dies, the pipeline will gracefully shutdown. However, if ...
Alexander Mills's user avatar
0 votes
0 answers
51 views

How to regain script interactivity when `... | bash` best practices?

Let's say we have a simple script named question: #!/bin/bash read -rp "What's your name?" ans echo "Your name is $ans" Let's use cat for our example cat question | bash We ...
punkbit's user avatar
  • 111
1 vote
1 answer
439 views

zsh: check exit code of pipeline of commands

I have long pipeline of commands in zsh script: pv /dev/sda > sda.raw | sha256sum > sda.raw.sha256 | cut -c61-64 | read SHASUM how can I check inside an if statement, that all commands exited ...
Martin Vegter's user avatar
3 votes
2 answers
632 views

Piping and input stream redirection to same command, which input does command take?

I was trying to run the command given below cat numbers.txt | sort < words.txt The above command has a pipe from cat command feeding to input stream of sort command. But at the same time, input ...
Abhishek P G's user avatar
0 votes
1 answer
44 views

Buffering follow + automatic pagination of output

I have a utility that continuously outputs live logs from a program. But it can be quite hard to read the logs lines as they scroll past me in the terminal. Ideally it should scroll to one full page, ...
David S.'s user avatar
  • 101
0 votes
0 answers
27 views

Bash equivalent of zsh alias at end of command (for pipes) [duplicate]

In zsh, I can set an alias at the end of a command, e.g. making G pipe through grep or L pipe through less. Can I do something similar in bash? E.g. ls G key would expand to ls | grep key or ls L ...
SRobertJames's user avatar
0 votes
1 answer
210 views

When simply `| cat -`, I get file descriptor errors

Here's the working diff command: $ diff -u <(echo 'foo:bar:baz' | tr : "\n") <(echo 'foo:baz' | tr : "\n") --- /dev/fd/11 2023-08-30 13:11:50 +++ /dev/fd/13 2023-08-30 13:...
Scoobie's user avatar
  • 101
0 votes
1 answer
433 views

Continuously polling a named (FIFO) pipe

I am trying to use a named pipe to monitor the activity of a running shell script. That shell script will write to the pipe that it has "Started" processing. When it has completed ...
nbmlane's user avatar
2 votes
0 answers
84 views

Waiting until an application is reading (over a TTY)

I'm writing an expect-style program to interact with a command-line application on Linux. Is there any way that my program can know when the application on the other side of the PTY tries to perform a ...
Subhav's user avatar
  • 21
9 votes
1 answer
733 views

Exit status of pipe with timeout command behaves differently in interactive shell vs shell script

Interactive sh shell session: $ sh $ timeout 1 yes | sed -n s/a/b/p ; echo $? Terminated 143 $ Non-interactive script via sh -c: $ sh -c 'timeout 1 yes | sed -n s/a/b/p ; echo $?' 0 $ Why do those ...
finefoot's user avatar
  • 3,142
-1 votes
2 answers
249 views

Difference in bash 3 between pipe and here-string

If I run: IFS=':' cat <<< $(echo $PATH) then the ":" are used for word splitting and the output contains spaces instead of ":", but if I run: echo $PATH | IFS=':' cat ...
Allen's user avatar
  • 206
0 votes
0 answers
100 views

mc command hangs with --version parameter when writing to stdout but works when piping to cat?

I observed odd behaviour of a utility binary (it's actually the minio client mc in a recent-ish version) when running it from the command line. If I just run it like ./mc --version, the command hangs ...
BMBM's user avatar
  • 147
2 votes
1 answer
164 views

What are the security implications of piping apt output compared to apt-get?

I tried to install all python3 packages so that I don't have to manually go to terminal and install one everytime. However due to some dependency issues apt refused to proceed: sonic@boomboom:~$ sudo ...
kseo6322's user avatar
0 votes
1 answer
101 views

Pipe : gunzip files and concatenate

I feel like this should be simple. I have a directory with >1000 gzip-ed files. I'd like to gunzip them all and concatenate. Running gzip -dk *.gz | cat > output.file produces the following ...
ThePresident's user avatar
0 votes
1 answer
462 views

Continuously display last line of command output without clearing the screen

I want to print only the last line of the output of a command on the same line continuously without clearing the screen. This would be part of another function. Let's say my script has the following ...
blue's user avatar
  • 1
2 votes
1 answer
334 views

Creating PKCS#12 container from input from stdin

I want to create a PKCS#12 (a.k.a PFX) container for a private key and a certificate using openssl. My key and cert are contained in a single (unecnrypted) file: $ cat d.pem -----BEGIN CERTIFICATE-----...
mat's user avatar
  • 298
0 votes
0 answers
56 views

how does data flows through pipe in linux when sorting?

suppose I do something like cat afile [ | possibly awk or similar to process the file] | sort I know data flows something like afile -- stdout -- os buffer -- stdin -- sort When the size of afile is ...
eugene's user avatar
  • 101
0 votes
0 answers
26 views

Capture stderr of program into a variable without affecting stdout and interactive features [duplicate]

I have a program that asks the user for input and provides interactive features such as moving the cursor, deleting text, and more. After the user completes entering the input, the program clears the ...
sudormrfbin's user avatar
0 votes
1 answer
144 views

Piping output to tee breaks the conditional

I am trying to test for the presence of custom firewall-rules on servers. I am doing this by processing the iptables output with awk and returning an appropriate exit-code. This works as intended, and ...
ARMs_for_the_poor's user avatar
0 votes
1 answer
190 views

bash: clarification on order of redirection and pipes

In an attempt to further understand bash, I am trying to replicate some functionalities of it. Right now I am in the process to implementing the redirecting and pipeline features. To further ...
Fernando Gómez de la Varga's user avatar
1 vote
2 answers
80 views

Is there a program that concatenates non-seekable streams (size not known in advance) and can separate them again?

I'm trying to concatenate multiple input files/streams into one stream (using the imaginary command stream-cat), pipe that stream into ssh and on the remote host separate it back into individual files/...
David Scherfgen's user avatar

1
2 3 4 5
37