A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

learn more… | top users | synonyms

2
votes
1answer
95 views

Pipeline each line of a file to a specific variable in a second command

I want to execute the following instruction: basex -bword=ENTRY consulta.xq But for every line of a SampleText file (plain text mainly). The SampleText contains the following lines: hello evening ...
11
votes
1answer
10k views

Piping search term (not filename) to grep

I want to pick out certain lines containing a given number from a file. The file I want to search is called os_clusters/piRNA_clusters.bed. awk '{if (a[$0]++ == 0) {split($0,b,"."); ;split(b[1],c,"r")...
5
votes
3answers
485 views

Can you pipe to a .bash_profile function?

I received a great function for highlighting files in Apple's finder using the command-line. It's basically a wrapper for osascript. I got it from Mac OS X: How to change the color label of files ...
14
votes
2answers
10k views

How to download an archive and extract it without saving the archive to disk?

I'd like to download, and extract an archive under a given directory. Here is how I've been doing it so far: wget http://downloads.mysql.com/source/dbt2-0.37.50.3.tar.gz tar zxf dbt2-0.37.50.3.tar.gz ...
2
votes
3answers
329 views

Expand placeholders with same-named environment vars in text file?

There are a lot of variations on this question, but I wanted to ask it here anyways because I am so close (!) to finding a solution on this one - I just need a little nudge. Scenario I have several ...
11
votes
3answers
39k views

What makes a Unix process die with Broken pipe?

Here are some options I thought of, not sure which is the right one. There was an I/O error reading from the pipe. The process writing to the other end of the pipe died with a failure. All processes ...
1
vote
1answer
116 views

strange behaviour of strings command [duplicate]

I am trying to capture mysql traffic and pass those traffic to strings command as follows: tcpdump -i any -s 0 -l -w - dst port 3306 | strings This is working as expected and printing all mysql ...
1
vote
2answers
501 views

How to run chrome from command line by using process substitution?

I am using the following command without success - $ google-chrome <(grep "simple" SimpleDoc.txt) If I simply do a grep, I get - $ grep "simple" SimpleDoc.txt Very simple doc that contains ...
3
votes
2answers
4k views

Why does a named pipe not get deleted after system restart?

As far as I understand, named pipes are not written to disk but are stored in memory. Here's how I created a named pipe - $ mkfifo pipe21 $ grep "simple" SimpleDoc.txt > pipe21 & [1] 2775 $ ...
8
votes
4answers
5k views

Pipe the output of a command if it is successful

INPUT_FILE=`ls -rt $MY_DIR/FILE.*.xml | head -1 | xargs basename` I wanted to execute the second command (head -1) only if the first command is successful. How do I improve this command?
5
votes
2answers
632 views

Reusing pipe data for different commands

I would like to use the same pipe for different applications, like in: cat my_file | { cmd1 cmd2 cmd3 } Cmd1 should consume part of the input. Cmd2 should consume another part and so on. ...
3
votes
3answers
379 views

How to send part of the data through the pipe/file and later interactively?

I have a command I want to run, let's call it program. It is an interactive program similar to bash or python REPL. I want to start this program with some predefined input. I tried program << "...
1
vote
1answer
881 views

pipe curl colored output

this is the script color #!/bin/bash ESC_SEQ="\x1b[" COL_RESET=$ESC_SEQ"39;49;00m" case $1 in red ) COL_START=$ESC_SEQ"31;01m" ;; green) COL_START=$ESC_SEQ"32;01m" ;; yellow )...
2
votes
1answer
529 views

How can I work around the program failing if there is *any* stdin?

For example, the following works fine: /usr/bin/program It produces some output, and gets to result. But if I invoke it like this: echo -n | /usr/bin/program or this echo -n | bash -c "/usr/...
4
votes
1answer
2k views

How can I use tee and logger -t “some tag:”?

I've seen in another post which shows how to use tee and logger as shown below. tee >(logger) <<< "System Load is OK : $Current_loadadv" >(logger) is bash syntax to create a file ...
3
votes
5answers
708 views

How can I split a pipe into 16 byte sequences?

I'm writing a bash script to automate the decryption of a certain file; however the file is encrypted in 16 byte blocks, rather than simply encoded in its entirety. Here's some psuedo code to briefly ...
4
votes
3answers
1k views

How to capture stdout of a running process redirected to /dev/null

I want to capture stdout of running process whose stdout is being redirected to /dev/null. I tried using system call fcntl(), however the moment I close fd "1", it was being allocated for other files ...
5
votes
2answers
3k views

Problem with pipes. Pipe terminates when reader done

I am on OSX, using bash, trying to make sense of pipes. I wish to let a program communicate in two directions with a bash shell. I want to set this up in such a way that this is always the same shell, ...
3
votes
2answers
1k views

Does named pipe modify the filesystem?

As title. If I create a named pipe and then read/write on it, is the filesystem where the named pipe resides affected? I.e. is the data buffered on the filesystem until read, or does it reside in ...
0
votes
1answer
373 views

What to do when result from pipe is not last argument in next command

The example below shows what I want to do, namely take the result from my last pipe and put it into the middle of the next command (where I wrote $RESULT_FROM_FIND_COMMAND). $ find . -name "*0.01*txt"...
2
votes
1answer
2k views

Bash pipe output to more

I am wondering about some bash environment setting here: is it possible to set stdout of bash as a pipe to /bin/more? You know, like using vi as editor for your commands it should be possible to avoid ...
4
votes
2answers
244 views

Why do we need two file descriptors when creating an unnamed pipe?

I've been reading about unnamed pipes and as I understood them they're implemented as a buffer in memory. When creating the pipe I need to pass an array of size two and it returns two pointers (file ...
3
votes
2answers
875 views

OpenSSL buffering problem

I have a stream that I would like to encrypt in real time, but the problem is that openssl encrypt only when it could read from stdin enough bytes, otherwise it waits. Look at those examples: $ (...
2
votes
1answer
1k views

broken pipe error with popen and JS ffi

I am using a ffi for nodejs which for the most part has nothing to do with this question which is really about understanding pipes better, but does offer some context function exec(cmd) { var ...
1
vote
3answers
106 views

Variable scope in multiple pipes

I was trying to do something following, find . -name "*.dat" | get the basename of file | move filename returned by first command to basename returned by second command To give a concrete example, ...
13
votes
4answers
12k views

Are FIFO, pipe & Unix domain socket the same thing in Linux kernel?

I heard that FIFOs are named pipes. And they have exactly the same semantics. On the other hand, I think Unix domain socket is quite similar to pipe (although I've never made use of it). So I wonder ...
7
votes
1answer
340 views

“w | tail” chops text at right margin (78 characters)

Summary: w | tail chops rows after 78th column. When I run the "w" command, it works as expected. However, when I pipe the output to "tail", it chops the output to 78 columns wide (truncates anything ...
6
votes
3answers
2k views

Can I pipe any two processes to each other?

In this page from The Design and Implementation of the 4.4BSD Operating System, it is said that: A major difference between pipes and sockets is that pipes require a common parent process to set ...
2
votes
1answer
3k views

redirection to file issues with nohup and pipe

For the following command: ssh -t esolve@hostname 'sudo nohup bash -c "ls > log 2>&1 &"' I always got error information: nohup: ignoring input and appending output to `nohup....
2
votes
1answer
98 views

process continous output of synclient

I am using synclient to track the position of finger on the touchpad. I use the following command. synclient -m 100 | awk '{print $2,$3}' This command gives the 'x' and 'y' co-ordinate on the touch-...
4
votes
2answers
114 views

Who cuts off first symbol of process substituted file and for what reason?

$ type 1.sh #!/bin/bash -eu php <(echo 12) $ ./1.sh 2 $ type 2.sh #!/bin/bash -eu cat <(echo 12) $ ./2.sh 12 $ type 3.sh #!/bin/bash -eu echo 12 | php $ ./3.sh 12 $ type 4.sh #!/bin/bash -eu ...
3
votes
1answer
425 views

How can I copy a file between server using bsdtar?

I'm looking for the fastest way of copying a file between 2 linux servers using bsdtar. With normal tar, I would do something like: ssh root@remote 'tar -cz -C /my/path/ big_file.txt' | tar -zxv ...
0
votes
2answers
887 views

Redirecting stdin with stdout to file

So I have a program that takes in user input and outputs text based on the input. EDIT2: I want to create a script that runs a C executable and the script feeds the C program input from a file and ...
4
votes
4answers
3k views

Getting stdin from a named pipe

What I am trying to do is run python in a terminal window and redirect it's stdin from a named pipe. Then I write to the named pipe in another terminal and have that command execute on python. ...
4
votes
1answer
2k views

Pass colors from ls through pipe to awk 'print' statement

This is a follow-up to my question from yesterday, Show sum of file sizes in directory listing. Thanks to Zero Piraeus and a point in the right direction by Mauritz Hansen, I now have function dir(...
3
votes
3answers
1k views

Can't process stdout with pipe as it comes

I'm running tshark on a fifo, and the following is a bare example of a loop that prints the output of tshark as it comes: tshark -i $fifo | while read line; do echo $line done The problem ...
3
votes
2answers
466 views

How to duplicate a stream and process both parts in a streaming way?

Sometimes I want to insert something in a pipeline for reporting or some other secondary use. It might be as simple as wc -l, or a more complex beast like awk or even a python script. It'd be nice to ...
9
votes
4answers
10k views

Output to stdout and at the same time grep into a file

I have a script that outputs text to stdout. I want to see all this output in my terminal, and at the same time I want to filter some lines and save them in a file. Example: $ myscript Line A Line B ...
9
votes
1answer
3k views

background process pipe-input

if i want to display "aaa" on screen: (1)$: echo aaa | cat ... works OK (2)$: echo aaa | ( cat ) ... works OK (3)$: echo aaa | ( cat & ) ... NOT working (4)$:...
0
votes
3answers
1k views

pass piped variable to perl oneliner

if you have something like: find . -d -maxdepth 1 | wc -l | perl ... what ways do you have to tell perl to take this variable that's coming its way and substract 1? I try to substract 1 since find ...
4
votes
3answers
2k views

Does one end of a pipe have both read and write fd?

As far as I understood one end of a pipe has both read and write fd's and the other end also has read and write fd's. Thats why when we are writing using fd[1], we are closing the read end e.g. fd[0]...
4
votes
2answers
1k views

Read data from a pipe for a certain amount of time (in seconds)

Is there an easy way to create a pipe that only lasts for a given amount of wall time? I want to be able to take a "snapshot" of what's being written STDOUT for a period of time (rather than bytes or ...
0
votes
1answer
769 views

ps | grep shows bad output in subshell with jobs running in the background

I am using bash. I have a file named "a2draw" that contains only 1 line: sleep 99999 I start it using this command: bash a2draw & Now, I know and understand the trick with square bracket ...
2
votes
0answers
559 views

avconv/ffmpeg output through /dev/ttyAMA0

I tried to pipe streaming video from video4linux2 webcam connected to a Raspberry Pi through /dev/ttyAMA0 to a computer using an Arduino as a intermediary. First of all, is it possible? Because I was ...
38
votes
4answers
26k views

What are the advantages of using named pipe over unnamed pipe?

I was reviewing a set of interview questions that are asked from a unix admin; I found a topic called "named pipe". I googled the topic; to some extent I have been able to understand it :- named ...
4
votes
4answers
3k views

Races when piping two commands to a named pipe

I want to have one process reading from a named pipe that receives data from multiple sources: $ mkfifo /tmp/p But I can't figure out how to get it to work consistently. First Scenario - this ...
0
votes
1answer
448 views

Can someone provide an xargs example piping mysql query data into another command?

Can someone provide an example for the command xargs? I want to do a mysql query to return the ID field of a column then feed that result into xargs into another command say mysql query delete. How ...
1
vote
1answer
272 views

What rules govern the parent-child relationship of processes launched by shell across a pipe?

#!/bin/ksh # start_service: start the service my_server_executable 2>&1 | my_pipe_following_shell_script & exit 0 After I run the above start_service script from command line, it is ...
17
votes
1answer
5k views

What are guarantees for concurrent writes into a named pipe?

For example, I created a named pipe like the following: mknod myPipe p And I read from it from some process (for example, some server). For example purposes, I used tail: tail -f myPipe If ...
4
votes
5answers
11k views

Piping output to text file within a for loop

I'm trying to do the following within a for loop: Find files that satisfy a condition Echo the name of the files to a log file. Gzip the file. I can get the script to find the files and echo their ...