A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

learn more… | top users | synonyms

0
votes
2answers
21 views

Bash - Redirect output to a variable or file descriptor, then read from the variable or file descriptor

Here's an example of a bash script that redirects all output to a file (and shows output on the screen too): # writing to it exec > >(tee --ignore-interrupts ./log) exec 2>&1 echo "here ...
1
vote
2answers
40 views

cryptsetup: Attaching loopback device failed

I am using cryptsetup with detached header. For that I am using the --header parameter, to specify header file. I can open my device using: cryptsetup luksOpen /dev/sdb1 sdb1 --header LUKS-HEADER ...
1
vote
1answer
42 views

patch: “bad file descriptor” in combination with pipe and LD_PRELOAD/ libtrash

Very short summary: cat "$file" | patch produces a Bad file descriptor-error, but patch < "$file" works, when libtrash is preloaded with LD_PRELOAD, and only on bigger projects. Description: For ...
13
votes
3answers
3k views

How can I redirect `time` output and command output to the same pipe?

Suppose I have a binary called foo. If I want to redirect the output of foo to some other process bar, I could write ./foo | bar. On the other hand, if I wanted to time foo, and redirect the output ...
24
votes
14answers
27k views

What's a good example of piping commands together?

If you were helping someone to learn the concept of pipes on the command line what example would you use? The example that actually came up was as follows: cat whatever.txt | less I feel like ...
0
votes
1answer
29 views

what is the correct form for “tex file.tex | dvips | ps2pdf”?

I would like to streamline 3 commands into one, but I keep getting error message. tex | dvips | ps2pdf file.tex or possibly tex file.tex | dvips | ps2pdf Then divps says there is no file. Or I ...
1
vote
3answers
30 views

How do I organize and cut this portion of a directory into a file?

I'm new to unix and I'm not sure how to name my title but my main objective is to copy the contents of a directory and send it to a file. Then cut out the directory location to just have the name. ...
3
votes
2answers
44 views

Read a password securely from within process substitution?

Related: Securely feeding a program with a password How to read from stdin in process substitution? Process substitution and cat: can't read stdin I've read the above questions and I still ...
3
votes
2answers
68 views

Trace a binary stream from a device file

I'm debugging a binary stream that is coming from a device file. I would like to have the output printed out in real time as hex values. So far I've used tail -f /dev/ttyAPP2 | hexdump -C but after ...
3
votes
2answers
23 views

Piped commands performance in embedded Linux

Recently, we purchased a software solution to integrate into a device we're developing. There's plenty we need to modify and adapt to our needs, so today I was going through part of the code to see ...
1
vote
2answers
42 views

How to run a program in a screen, redirect all output to a file and detach

Running a command in a screen and detaching is quite simple. screen -S test -d -m echo "output of command that runs forever" However I would also like to pipe all the output to a file for logging, ...
6
votes
1answer
353 views

Using 'ping', 'cut' and 'tee' together

So, I understand that by using 'tee' I can redirect the output of a command such as 'ping' to stdout as well as a file. For example: > ping google.com | tee somefile This would display the ping ...
3
votes
3answers
39 views

How to pipe data from one command to another command which allows only a file as an argument?

I have some arbitrary data as an output of a command (e.g. a fragment of a binary data put out using tail). I want to feed it to another command/program which allows only a file as an argument (e.g. ...
2
votes
2answers
6k views

What's the difference between STDIN and arguments passed to command?

I could use the either form to execute the cat method: cat file_name cat < file_name The result is the same Then I want to execute man in the format of stdin man < file_name While ...
0
votes
3answers
50 views

piping with grep

I want to search "C" in the grades column inside the file "input.csv": roll,name,total,rank,grade 001,A B,78,23,C 003,F YZ,100,1,* and what I'm trying is: cut -d"," -f4 input.csv | grep "C" ...
0
votes
2answers
37 views

getting a list of all lowercase words in a file

I'm trying to get a list of all lowercase words in a file. This far I've gotten to the point of getting a list with the command line cat filename.txt | tr ' ' \\n | grep -w '[[:lower:]]*' | sort -u | ...
1
vote
3answers
55 views

Can I redirect the output of a program to cat with a pipe?

I have a folder with three files and basic contents: $ tail * ==> file1 <== file 1 contents ==> file2 <== file 2 contents ==> file3 <== file 3 contents I would like to see the ...
1
vote
2answers
114 views

How to redirect output of a program to a ZIP file? [closed]

I am running a script which is collecting a log of a server. I need to redirect this logs to a ZIP file. Right now I am collecting data into text file. How can I redirect it directly to ZIP?
1
vote
0answers
17 views

Half-close netcat connection

How can I have a netcat connection terminate the sending half of a TCP connection if its input reaches EOF? I have a (non-standard) TCP service which reads all its input (i.e. until the client sends ...
8
votes
4answers
329 views

How to use sed to manipulate continuously streaming output?

I am putting together a presentation for a non-technical audience. I have a program running in bash that outputs a continuous stream of values, a few of which are important. I would like to highlight ...
14
votes
3answers
1k views

In `dmesg | head`, (how) is dmesg being killed after 10 lines of output? [duplicate]

If I run these commands: dmesg | head -n 10 I presume the OS sends back some kind of signal to dmesg once head has read 10 lines. How does this work? What does head tell the kernel? This is ...
8
votes
3answers
31k views

What makes a Unix process die with Broken pipe?

Here are some options I thought of, not sure which is the right one. There was an I/O error reading from the pipe. The process writing to the other end of the pipe died with a failure. All processes ...
3
votes
2answers
33 views

Why builtin times doesn't work in pipeline?

I've the following sample script: #!/usr/bin/env bash sleep 5 printf "times cmd: " times printf "pipeline: " times | ( read user sys; echo $user; ) printf "head: " times | head -n1 printf "times cmd ...
5
votes
2answers
45 views

Understanding i/o redirection in the context of _process substitution_

Running GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu). I don't really understand process substitution (ProcSub) from the perspective of a user interested in lifting the hood on i/o ...
15
votes
4answers
12k views

How to check if pipe is empty

I have piped line in bash script and want to check if pipe has data, before feeding program Searching I found about test -t 0 but it doesn't work here. Always returns false. So how to be sure that ...
1
vote
2answers
17 views

Find unique lines based on only a section

I am wondering if there is a way to split lines by some character (E.g. :) and find unique lines based on one part of this split. For example if I had the following piped in via stdin file1: line 4 ...
1
vote
1answer
27 views

Broken pipe error when reading stops

I'm trying to make a quick & dirty bash function to compare the audio content of two FLAC files without decoding them into a temp file. With the flac command line tool I'm decoding each file into ...
2
votes
2answers
64 views

How to sort data stream by 2 column names (column number may vary)?

I get a stream of data from an API with which look like this: redID blueID whiteID 1 22 2 44 15 41 2 15 15 31 2 14 What I need to do, is to sort ...
7
votes
4answers
331 views

Split an input for different command and combine the result

I know how to combine the result of different command paste -t',' <(commanda) <(commandb) I know pipe same input to different command cat myfile | tee >(commanda) >(commandb) Now ...
1
vote
1answer
41 views

How to provoke a broken pipe

I try to debug a python script that interfaces MPD using python-mpd2. When the script tries to perform an action after being idle for a couple of hours, it terminates with an exception ...
4
votes
3answers
3k views

Why piping find and grep returns nothing?

I'm trying to have grep search inside specified files that are returned by find: find . -type d -name 'mydir*' -exec find '{}' -name '*.java' \; | grep 'MyClass' This doesn't work. Meanwhile, ...
1
vote
1answer
52 views

Get absolute path of file OS X

I am creating a script which will prompt the user to enter a location of file and it will convert it to an absolute path. This is what I have so far. I'm not sure how to append these together. ...
2
votes
1answer
28 views

Appending to a file suffix range with echo and tee

I am unable to figure out how to properly pipe and tee to append to a range of files. Instead of manually typing them out echo "help!" | tee >> file1 file2 file3 I am trying to figure out: ...
0
votes
0answers
8 views

bash one-liner for opening `less` on the last screen w/o temporary files [migrated]

I try to create o one-liner for opening less on the last screen of an multi-screen output coming from standard input. The reason for this is that I am working on a program that produces a long AST and ...
4
votes
1answer
99 views

Ctrl-c kills shell when pipeline is used in input redirection subshell

If I run a command like: cat <(echo 1 | pv) | pv cat <(echo 1 | pv) | less cat <(echo 1 | pv) | cat The command seems to run forever. Entering ^C (SIGINT) kills the entire shell instead ...
36
votes
4answers
2k views

Should I care about unnecessary cats?

A lot of command-line utilities can take their input either from a pipe or as a filename argument. For long shell scripts, I find starting the chain off with a cat makes it more readable, especially ...
118
votes
15answers
47k views

Get exit status of process that's piped to another

I have two processes foo and bar, connected with a pipe: $ foo | bar bar always exits 0; I'm interested in the exit code of foo. Is there any way to get at it?
11
votes
5answers
1k views

Utility to buffer an unbounded amount of data in a pipeline?

Is there a utility that I can stick in a pipeline to decouple read and write speeds? $ producer | buf | consumer Basically, I want a utility buf that reads its input as fast as possible, storing it ...
3
votes
3answers
114 views

What to use instead of `buffer` for 1GB buffer? [duplicate]

buffer(1) seems to be old-ish and have hard-coded values preventing it to cache large amount of data. $ buffer -m 1G max_shmem 1 too low // it doesn't even understand gigabytes $ buffer -m 1000M ...
1
vote
1answer
40 views

Unix awk decimal export in pipe

I'm struggling to adapt a script, which loops through a large number of individual files (subj_id), and prints out the value of $4 in a specified line. Text file contains lines # Measure Brain, ICV, ...
20
votes
6answers
23k views

how do I trim each line in bash?

I would like to remove all leading and trailing spaces and tabs from each line in an output. Is there a simple tool like trim I could pipe my output into?
2
votes
1answer
29 views

gnuplot output of a pipe

Inspired by this question I tried to pipe output to gnuplot and then plot it. My pipe is: cat file.txt | grep CU | perl -e 'while(<>){print +(split)[3], "\n"}' | gnuplot file.gp Where ...
1
vote
1answer
29 views

Find files and send to a temp file

Vim does support a nice option to accept a file with list of files that are then opened in a Quickfix list. Rough example, I find all files in ~ that are smaller than 1KB and pass this to Vim: find ~ ...
19
votes
9answers
4k views

Grep lines but let the first line through [duplicate]

Consider the following output from df. Filesystem Size Used Avail Use% Mounted on /dev/sda1 23G 6.1G 16G 29% / udev 10M 0 10M 0% /dev tmpfs 397M 420K ...
16
votes
4answers
716 views

How to trick a command into thinking its output is going to a terminal

Given a command that changes its behaviour when its output is going to a terminal (e.g. produce coloured output), how can that output be redirected in a pipeline while preserving the changed ...
3
votes
3answers
436 views

How do I pipe the output of tar through mv?

A previous question of mine asked how to pipe downloaded files through tar, now I would like to know how to pipe the output of tar through mv. See I have this command at the moment: wget -c ...
4
votes
4answers
123 views

Use awk interactively through a pipe

Looking at this question, I've noticed that awk can't read user input if the file is being piped into standard input, but it does behave as expected if reading input from a file given as a command ...
-1
votes
1answer
50 views

are named pipes (mkfifo) the predecessor of RabbitMQ? [closed]

Is RabbitMQ, for inter process communication, like pipes and named pipes? How does RabbitMQ compare to named pipes? Except distributed systems. (RabbitMQ, for those who haven't encountered it, is an ...
3
votes
2answers
100 views

Process each line of output from `ping` immediately in pipeline

I have a few examples of different ways of extracting timing information from ping -c 10 google.com. For some of these pipelines, a line of output is produces every so often, just like the output of ...
7
votes
4answers
1k views

Get PID of any command in backgrounded piped sequence of commands

If, in bash, I execute: cmd1 | cmd2 | ... | cmdi | ... | cmdn & where the cmd{1..n} may not be distinct, how do I get the PID of cmdi? Alternatively, how can I signal the cmdi process? (For ...