Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [pipe]

A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in <unistd.h>. Shells provide pipe creation between processes using "|".

0
votes
2answers
37 views

How do I use command line arguments passed to a shell script that uses xargs?

I have a script that will iterate a set of git checkouts and run a git command in each directory. I now want to be able to pass a parameter to the shell script from the command line and have that ...
0
votes
1answer
43 views

Writing continuously to named pipe

I have a problem that I can't seem to solve involving two named pipes. Basically, I have two processes that I don't have control over. They are communicating via named pipes (mknod pipe p). Process ...
2
votes
0answers
11 views

Avoid pipeline stalls from write caching to usb drive

I have the following process I use for backups: btrfs send snapshots/home-2016-06-04 | zstd --verbose -T4 - | gpg --batch --passphrase-file /tmp/secret --compress-algo none --symmetric - > /mnt/...
0
votes
1answer
33 views

Run program when/instead of writing to FIFO? [on hold]

I have a program that writes data every second to a FIFO. Now I want to alter some of this data and write it to another FIFO. What would be the best approach? Can I somehow pipe this directly to my ...
1
vote
1answer
63 views

Exit status of chained commands

I want to run a potentially invalid command in a variable cmd. I can't afford to have it just fail me and retrieve the exit status. What I can try to do is pipe it to : and then I'll have a successful ...
0
votes
1answer
25 views

Find files and specify some to come first in playlist, and the rest in random order?

The following code creates a list of the files in the directory and then shuffles them in a random order then spits out the file in a text document called list.txt find *.EXT | sed 's:\ :\\\ :g' | ...
1
vote
1answer
27 views

`Convert` and standard output

I am learning how to use terminal and as an exercise tried to convert a bunch of .png images to ASCII-art using convert and jp2a with this command: convert test.png -format jpg - | jp2a - This ...
2
votes
1answer
34 views

Difference between Piping and Reading Input

What is the difference between cmd1 > tmp && cmd2 < tmp and cmd1 | cmd2, besides the creation of a tmp file (obviously) ? If my comprehension of pipe is right, I expect cmd2 to read ...
2
votes
2answers
50 views

PIPESTATUS from commands in $(…|…)

How to read ${PIPESTATUS[0] of a command, when a variable is set in current shell? Is there a way to pass it to current shell somehow? I am setting variable MAIL=$(ldapsearch.... | find_for_mail) and ...
1
vote
2answers
31 views

Non-Root User SSH Privileged Command with File Piping

I have an a SSH server where root login is disabled. I'm hoping to run a privileged command that will want input from a file I have after being executed. I'm hoping to do all without user interaction ...
2
votes
1answer
71 views

What is the most succinct way of terminating the rest of a pipeline if a command fails?

Consider the following: command1 | command2 | command3 As I understand pipelines every command is run regardless of any errors which may occur. When a command returns stderr, it is not piped to the ...
0
votes
2answers
90 views

How to feed data through pipe into utility/command as part of shell here-doc?

The following example is kind of classical way of using pipes. On right side of pipe we have a "simple" utility which reads data from stdin (pipe) and prints to stdout in reverse order: { cat <<...
3
votes
1answer
53 views

Exit status of command in pipeline

I have come across this code but cannot understand exec >&p edit 1 to my understanding : #! /usr/bin/ksh exec 4>&1 ## standard output is first saved as file descriptor 4 ...
0
votes
0answers
40 views

How does bash implement the “|” operator? [duplicate]

I have a question regarding the pipe operator ("|"). How does bash connect the output of one command or process to the input of another? Is it using the pipe system call or some other way? A sample ...
4
votes
2answers
132 views

Inner for loop when run in background in bash spawns new bash process

I am executing below script LOGDIR=~/curl_result_$(date |tr ' :' '_') mkdir $LOGDIR for THREADNO in $(seq 20) do for REQNO in $(seq 20) do time curl --verbose -sS http://dummy.restapiexample.com/...
11
votes
3answers
380 views

How to make a pipeline wait for end-of-file or stop after an error?

I tried the following command after watch this video on pipe shenanigans. man -k . | dmenu -l 20 | awk '{print $1}' | xargs -r man -Tpdf | zathura - It basically prints a list of manpages to dmenu ...
9
votes
1answer
577 views

How does a program know if stdout is connected to a terminal or a pipe?

I'm having trouble debugging a segfaulting program because the ouput right before the segfault is what I need, but this is lost if I'm piping the output to a file. According to this answer: https://...
2
votes
2answers
60 views

What's the POSIX specification on behavior of built-in commands with redirections and/or piping?

Consider the following commands: exit > /dev/null exit | cat On a few shells (ksh, bash, (d)ash), the behavior is the same: The first command causes the shell to quit immediately, while the ...
0
votes
2answers
42 views

Pipe and filter bash outputs in realtime

I am running some commands in bash (basically some ffmpeg commands), that I'm using grep and awk to filter out the results. The command takes some time and continously outputs some results as it ...
1
vote
0answers
71 views

Broken pipe error [closed]

The following are my 2 Rscripts written as mapreduce code- 1) mapper.R #! usr/bin/env/ Rscript # mapper.R - Wordcount Program in R # script for Mapper (R-Hadoop Integration) trimWhiteSpace<- ...
16
votes
2answers
1k views

Understanding piped commands in Unix/Linux

I have two simple programs: A and B. A would run first, then B gets the “stdout” of A and uses it as its “stdin”. Assume I am using a GNU/Linux operating system and the simplest possible way to do ...
2
votes
1answer
65 views

Run multiple commands after pipe

I am piping streamer ouput to mpv player which works fine. streamer -o - | mpv - But before starting mpv , I want to kill one program and then pipe to mpv. streamer program takes some time to ...
1
vote
1answer
69 views

Why can't I chown a pipe? [closed]

From a discussion on useless use of cat. I decided to see if It was possible to avoid some of the troubles by changing the owner of the pipe. So I did: # cat | cat & [1] 16500 # cd /proc/16500/...
1
vote
1answer
40 views

Pipe stderr only if a terminal application crashes

I have a program which interacts with the user on the terminal. But if it crashes I'd like to pipe the stderr through another tool to tidy / prettyprint it. Is there a way to do this? As far as I ...
-1
votes
2answers
42 views

Only command after pipe executes, how does the pipe work in these scenarios?

What happens in the following two examples? Why is only the second command in each pipeline executed? How does the pipe work here? example 1 $ cat /var/log/messages | date Mon Apr 8 17:10:02 IST ...
3
votes
1answer
70 views

Make GNU Parallel not delay before executing arguments from STDIN

GNU Parallel, without any command line options, allows you to easily parallelize a command whose last argument is determined by a line of STDIN: $ seq 3 | parallel echo 2 1 3 Note that parallel does ...
2
votes
1answer
42 views

Cat blocking when piping output from /usr/bin/gssproxy (`timeout -k 1 1 /usr/bin/gssproxy | cat` does not terminate)

I use gssproxy under Arch Linux, in particular Name : gssproxy Version : 0.8.0-1 I encountered the following behavior: When trying to pipe the output of /usr/bin/gssproxy like ...
2
votes
1answer
66 views

Why isn't it possible to read from `stdin` with `read` when piping a script to bash?

I'm not looking for work-arounds or solutions for the issue. I'm fine with it not working like that in bash. I just don't understand why it doesn't work. I'm looking for an in-depth answer why the ...
0
votes
0answers
59 views

Bandit0 'Broken Pipe Error' Problem

Tried to connect with ssh -p 2220 bandit0@bandit.labs.overthewire.org and after successfully entering the password, I'm met with this error: packet_write_wait: Connection to 176.9.9.172 port 2220: ...
0
votes
1answer
74 views

Read paths on stdin and spawn a new interactive shell for each line

Consider a command that searches the entire home directory for a file or directory with the wrong permissions: $ find $HOME -perm 777 This is just an example; the command might be listing broken ...
2
votes
2answers
28 views

Check which strings from list never appear in recursive search of directoy

I have a list of strings stringList, where I want to recursively search a directory and find all of the strings in stringList that did not appear in any search result. Right now, I have cat ...
3
votes
2answers
60 views

Output redirection is not working with top piped to grep

When I run this command in Linux: $ top -b -d 20 | grep "load average" -A 20 > top.log top.log is always empty. But when I run this command: $ top -b -d 20 | grep "load average" -A 20 | tee top....
1
vote
1answer
57 views

Bash: child process and pipelining

The purpose of my script is to count how many lines there are in a file. I know I could use wc for example but the purpose of this exercise is to understand processes and pipelines in Linux. The ...
0
votes
1answer
75 views

How to run shell command inside awk and redirect to another shell command?

I want to get particular string from the files downloaded using curl and redirect the downloaded file content to another shell program. Here is the code : curl "http://file-list-url" -s | awk 'BEGIN {...
2
votes
1answer
57 views

Why does echo “a” | wc -m return 2? [duplicate]

Why does echo "a" | wc -m echo 2? Other variations in which I try to reduce the number of spaces yield the same result.
-1
votes
2answers
63 views

Pipe each batch of xargs trough wc -l

So my task is to find the file with the most hardlinks in a directory. So far i have : find . -name "file*" | xargs -I{} -n 1 find . -samefile {} which gives me: ./hardlinkFIle245 ./...
4
votes
2answers
299 views

pipe commands inside find -exec?

Let's suppose I want to find all .txt files and search for some string. I would do: find ./ -type f -name "*.txt" -exec egrep -iH 'something' '{}' \; What if I want to do a more complex filtering, ...
0
votes
1answer
82 views

echo, pipe and command substitution

I want to substitute an echo command followed with | (pipe). For example, $(echo "echo 'hello' | cat") returns 'hello' | cat I expect this to behave like echo 'hello' | cat which returns hello ...
1
vote
2answers
35 views

Two named PIPEs (PIPE_in/PIPE_out) connected with `tail -f` | String sent to PIPE_in doesn't reach PIPE_out

1.Create named PIPEs, pipe_in and pipe_out by running: $ mkfifo pipe_in $ mkfifo pipe_out 2.Connect pipe_in to pipe_out: TERM0: $ tail -f pipe_in > pipe_out 3.Send string hello world! to ...
0
votes
1answer
36 views

Getting the IP address of a given interface [duplicate]

I need the IP (v4) address of a device I specify. The response will be later used in a script. This is what I have: $ IFACE=eth1 $ ifconfig $IFACE |grep "inet " | awk '{print $2}' 10.0.0.33 Is there ...
0
votes
2answers
132 views

How to directly compress CURL output as named archive entry?

I've hundred of text files I need to CURL from remote host. Currently I've a loop which download each file and then compress it as new .zip entry. Is it possible to directly pipe CURL's output to an ...
0
votes
0answers
34 views

C++ script on Spark Cluster throws exit status 132

I'm trying to run a c++ program on spark cluster by using the rdd.pipe() operation but the executors throw: java.lang.IllegalStateException: Subprocess exited with status 132. The spark jar runs ...
1
vote
0answers
33 views

Getting awk value and outputting to a file [duplicate]

I need to get a value from a log file which I am getting using awk at the moment. I can get the number printing on the console but when I try to put that value into a new file nothing happens. Here is ...
0
votes
0answers
131 views

How to write Docker logs to a file in real time (à la `tail -f`)

I docker that output logs stdout stderr, which can be viewed using: docker logs -f $LOGS_CONTAINER_ID I also added 'sed', which puts the container id on each line: docker logs -f $LOGS_CONTAINER_ID ...
0
votes
1answer
36 views

Using bash script to pipe breakpoints to debugger prompt

I am using bash to write a script to make it easier to set breakpoints. I am trying to see if I can use echo and pipe to send set breakpoints commands to the java debugger jdb. The command I have ...
2
votes
2answers
75 views

Redirecting from right to left

From old habits, I'm always redirecting "from left to right", eg. cat file | bar foo | bar I've noticed you can redirect "from right to left". However, is it possible to do this for the second form (...
2
votes
3answers
193 views

Piping Multiple Numbers into Sed

Is it possible to pipe a list of numbers into sed? The pipeline I currently have looks something like: grep -nP 'foo' file_full.txt | sort | awk -F'[:;\t]' '{print $1,$3,$9,$13}' The output of ...
0
votes
1answer
135 views

Socat. Bridge TCP - SERIAL PORT. With log

I'm trying to create a bridge between TCP server and a client connected thorough a serial port using socat. I emulate my TCP server with the following command socat tcp-listen:8888,reuseaddr - I ...
0
votes
1answer
27 views

How to prompt data on piped commands [duplicate]

I'm trying to use this command: mysqldump --opt --hex-blob \ -h source -p xxxx -u user1 \ source_db | \ mysql \ -h target -p yyyy -u user2 \ target_db So, how can I rewrite it to ...
2
votes
1answer
66 views

Reverse the order of file matching as arguments

I have a program that reads files in a certain order. For this example, they all have the same extension, txt {program_name} {input 1...} {output} I happen to name the output file name "a.txt" so it ...