A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

learn more… | top users | synonyms

0
votes
0answers
33 views

How to do multi threading using mkfifo? [closed]

I have a file which having milions of records like below. 8350000014|1234567|NULL|NULL 7336004920|478637|NULL|NULL 9876006473|478632|NULL|NULL 6730000012|1234567|NULL|NULL ...
0
votes
0answers
31 views

When was pipelining introduced?

I understand early versions of Unix were licenced from AT&T to Universities and ran on PDP computers. I read in Tanenbaum's book (operating systems) that the first versions of Unix were only a ...
8
votes
1answer
71 views

Why doesn't file redirection to and from a named pipe work, but piping to cat does?

This is a simple echo server in Unix, using nc: mkfifo fifo cat fifo | nc -k -l 4458 -v | cat >fifo (based on this) As I can see it, the data flow works as follows: fifo (my named pipe) | | ...
4
votes
3answers
462 views

How do pipes and infinite streams work?

I usually use command1 | command2 | command3 a lot in Linux but most of them are dealing with definite content. When I tried this with an infinite stream cat | sed '' | sed '' which hopefully ...
16
votes
5answers
911 views

How to trick a command into thinking its output is going to a terminal

Given a command that changes its behaviour when its output is going to a terminal (e.g. produce coloured output), how can that output be redirected in a pipeline while preserving the changed ...
0
votes
1answer
28 views

how to color specific strings in standard output and standard error [duplicate]

How do you color specific strings in standard output and standard error? For example: gpg --verify Outputs either a good signature or a bad signature string. How can I color the good one, green; ...
1
vote
1answer
42 views

Persistent process substitutions? [duplicate]

Is there a way to take a process substitution like <(grep foo some/file), and save it to a "file-like entity" on disk (reminiscent of a fifo), so that, every time this thing was opened for reading ...
3
votes
2answers
37 views

Mark descriptor as terminal for the -t test

One can test if a descriptor is connected to terminal with -t test. if [ ! -t 0 ]; then echo "Stdin is not from terminal" fi Having a descriptor X, pointing to a file or to a process (with ...
0
votes
1answer
35 views

globbing and wildcards

Are wildcards ever supposed to be expanded for a program that is not the first in the pipeline? Every example uses wildcard for the first program in the pipelilne. For example, the following does ...
20
votes
4answers
1k views

How to invoke vim editor and pipe output to bash

Sometimes I need to write text and then pipe that text into another command. My usual workflow goes something like this: vim # I edit and save my file as file.txt cat file.txt | pandoc -o file.pdf # ...
1
vote
1answer
43 views

Filter out duplicate lines of tail -f [duplicate]

I often need to tail -f apache access logs for websites to troubleshoot issues- one thing that makes it annoying is that anyone loading a page once may cause 12+ lines to get written to the log, and ...
213
votes
11answers
86k views

Turn off buffering in pipe

I have a script which calls two commands: long_running_command | print_progress The long_running_command prints a progress but I'm unhappy with it. I'm using print_progress to make it more nice ...
2
votes
2answers
137 views

How do I double each line of piped output? [duplicate]

How do I double each line of input piped in? Example: echo "foobar" | MYSTERY_COMMAND foobar foobar
9
votes
4answers
10k views

Output to stdout and at the same time grep into a file

I have a script that outputs text to stdout. I want to see all this output in my terminal, and at the same time I want to filter some lines and save them in a file. Example: $ myscript Line A Line B ...
1
vote
0answers
7 views

Track amount of data application and a device file are exchanging

Is it possible to track how much data is going into a device file and an application as a third person? Background A web server is communicating with a device /dev/ttyACM0. The server offers a HTTP ...
0
votes
2answers
28 views

operate on different sections of stdin without intermediate file

Is there a simple way to do this with redirection or pipes without creating FILE1? I want to apply process2 to the body of the output of process1, without touching the first and last few lines. ...
0
votes
1answer
31 views

How to determine if it is a fault of the Network Card?

Basically, I have a PC having problems. Let's call it POO. POO is a pc running Gentoo Linux, with KDE. It is quite freshly setup on an old hardware. So far I do not notice it has other problems, ...
8
votes
5answers
459 views

Is there a tool to dynamically redirect output to a new file on request

I'm currently redirecting the output of a monitoring tool to a file, however what I'd like to do, is to redirect this output to a new file on my request (using a keybinding), without stopping the said ...
3
votes
2answers
31 views

Chaining Linux terminal commands

I have a function that has my EV3 speak speak(){ espeak -a 200 -s 130 -v la --stdout "$@" | aplay; } it works by simply speak "Say this" I want it to say the contents of a file, so I have this ...
8
votes
3answers
12k views

Why is my variable local one 'while read' loop, but not in another seemingly similar loop?

Why do I get different values for $x from the snippets below? #!/bin/bash x=1 echo fred>junk ; while read var ; do x=55 ; done <junk echo x=$x # x=55 .. I'd expect this result x=1 cat ...
0
votes
1answer
53 views

grep with piping and showing multiple lines

I am trying to use grep to search a large text file, and return the paragraph containing a few key words. I also want to return the surrounding lines in the result. So, for example I have the ...
5
votes
1answer
53 views

Cannot grep jobs list when jobs called in a function

I can grep the output of jobs, and I can grep the output of a function. But why can't I grep the output of jobs when it's in a function? $ # yes, i can grep jobs $ jobs [1]+ Running vim ...
8
votes
4answers
9k views

Why doesn't grep using pipe work here?

I have the following command: find / -name libGL.so.1 Which returns lots of lines with "Permission denied". I want to exclude such lines, so I added the following: find / -name libGL.so.1 | grep ...
2
votes
1answer
1k views

broken pipe error with popen and JS ffi

I am using a ffi for nodejs which for the most part has nothing to do with this question which is really about understanding pipes better, but does offer some context function exec(cmd) { var ...
6
votes
1answer
116 views

Extra output on input redirection

I have a folder with three files: $ ls a b c If I pipe the output of ls to wc, I get the right result: $ ls | wc -l 3 However, when I specify the input to wc as the output of ls, I get extra ...
4
votes
2answers
177 views

Running several times the same command with several arguments after a pipe

I have a key (of random binary data) that gets generated by get_key. And with this key, I can do several things with my encrypted files. For example, I can decrypt them. get_key | tee >(decrypt ...
26
votes
7answers
30k views

how do I trim each line in bash?

I would like to remove all leading and trailing spaces and tabs from each line in an output. Is there a simple tool like trim I could pipe my output into?
0
votes
0answers
7 views

Why can't I pipe my ruby script into lemonbar? [duplicate]

If I try to pipe this ruby script: while true puts "hi" sleep(1) end into lemonbar it does not work. Lemonbar displays nothing. If I run lemonbar manually and type in things, it works just ...
1
vote
2answers
56 views

Grep command using contents of a heredoc

I'm creating a script to update a remote computer and I want to know if any user is using the computer before I update. I have a list of users and I created a heredoc with all of them. The command I'm ...
2
votes
1answer
1k views

Linux nuisance: /dev/stdin doesn't work with sockets

Linux has this blitheringly annoying peculiarity that /dev/stdin won't work with sockets - it is hardcoded to return ENXIO. Try this: socat TCP-OPEN:localhost:1234 EXEC:cat\ /dev/stdin,nofork ...
4
votes
4answers
3k views

Getting stdin from a named pipe

What I am trying to do is run python in a terminal window and redirect it's stdin from a named pipe. Then I write to the named pipe in another terminal and have that command execute on python. ...
1
vote
1answer
24 views

Piping output between two simultaneous, independent programs

Let's say I have two programs which play chess: chess_1 and chess_2. They keep track of the board themselves, and take as input opposing moves, and then output their moves. For example, if I wanted to ...
0
votes
1answer
38 views

send signal to command pipe tears down whole pipe

I have a simple example of two processes the first is a simple loop which does some processing #!/bin/bash function signalHandler() { echo "sig: $1 received ==> exit" for i in {1..5}; do ...
0
votes
1answer
35 views

Cygwin - tail stops reading input pipe after first write

I have the following script to run my application in Ubuntu with redirecting its input (where $in is named pipe which is used to send text lines to stdin). tail -f "$in" | "$app_bin" $app_args & ...
1
vote
1answer
62 views

Check if the files in a specific directory have the proper extension?

I have created a script that will create a few directories and then organize other files from one directory by moving them into specific sub-directories based on their extension (i.e. .gif in media, ...
5
votes
4answers
889 views

Why does sed act differently depending on the output file?

If I run: cat messages.txt | sed -e 's/a/a/g' > messages.txt on one large file (2500+ lines) I find that the resulting file will only have about 900 lines after the command in cygwin and will ...
9
votes
1answer
333 views

Make program in a pipe think it has tty

Test environment: $ mkdir testdir $ cd testdir $ echo | tee "file name" Now, ls will print file?name (under Ubuntu GNU bash, at least), while ls | cat will print file and name in separate lines. ...
-3
votes
1answer
32 views

How to do inter process communication using single bidirectional pipe in linux?

How do 2 processes communicate by using single bidirectional pipe, such that both the ends of the pipe could read and write. I have an assignment to do, kindly help me with this.
15
votes
2answers
561 views

How do I use grep while keeping the original text color? [duplicate]

For example, I have a output: Hello, this is the output. (let's say that for example hello is colored red, and the is colored green, and output is colored purple). Now, let's say that this is the ...
3
votes
1answer
38 views

Standard/canonical way to test whether foregoing pipeline produced output?

The situation I have in mind has the following structure: % some_command | [PRODUCED OUTPUT] || echo 'no output' >&2 Here [PRODUCED OUTPUT] stands for some as-yet-unspecified testing ...
3
votes
2answers
71 views

avoid multiple pipes broken due to missing command

In shell scripts I often use multiple pipes like this: cmd1 | cmd2 | cmd3 Now if cmd2 is missing the pipe will result "broken" and cmd3 won't receive any input. As workaround I could test ...
19
votes
8answers
5k views

How to pipe output from one process to another but only execute if the first has output?

How can I rewrite this command to only email if there is output from the mailq | grep? mailq | egrep 'rejected|refused' -A 5 -B 5 | mail -s 'dd' email@email Is this even possible on one line? See ...
15
votes
4answers
13k views

Check if pipe is empty and run a command on the data if it isn't

I have piped a line in bash script and want to check if the pipe has data, before feeding it to a program. Searching I found about test -t 0 but it doesn't work here. Always returns false. So how to ...
2
votes
5answers
3k views

Why does the command shuf file > file leave an empty file?

I know this is sort of a duplicate of another question (Why this sort command gives me an empty file?) but I wanted to expand on the question in response to the answers given. The command shuf ...
1
vote
2answers
64 views

Read files from directory and organize them by extentsion into corresponding folders

I'm trying to create a bash function script that will allow me to create multiple directories with the names below. I'm also trying to organize files by extension into their corresponding folders ...
3
votes
2answers
42 views

pipe process output to command split by line and whitespace

On the system I'm using I often need to cancel jobs I run. I can check my jobs like this: $ showq|grep jquick 2001744 jquick Running 120 00:08:26 Mon Apr 11 11:06:02 2002020 ...
134
votes
15answers
54k views

Get exit status of process that's piped to another

I have two processes foo and bar, connected with a pipe: $ foo | bar bar always exits 0; I'm interested in the exit code of foo. Is there any way to get at it?
3
votes
4answers
55 views

Search within a search

I've gotten used to using grep for my command line searches and wanted to know how to successfully do a search using the result of another search. Here's my attempt, where I am looking for 'tool' ...
0
votes
1answer
34 views

tar: option requires an argument — 'f'

I am trying to download content by following this tutorial. But when I use the following code from the tutorial: curl https://start.spring.io/starter.tgz -d style=web -d name=simple | tar -xzvf ...
1
vote
1answer
47 views

How to get `yum list` output to stay on one line when getting output via remote ssh command?

When I SSH to a machine with ssh machine and run yum list it outputs everything that's installed as expected, and importantly every item is on one line like so: xz-devel.i386 ...