A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

learn more… | top users | synonyms

4
votes
3answers
1k views

cycle through one command's output and use in another command [duplicate]

How can I use the output of one command - line-by-line - into another command? I know how to do this with the | pipe symbol, but this uses the entire output in one command. I would like to go line by ...
1
vote
3answers
118 views

Automating pipes?

I find myself using basically the same line over and over again: cat file | command1 | command2 | command3 > file Is there a way I can put all these pipes into one script, so I can just run ...
11
votes
2answers
4k views

Piping data to a process's stdin without causing EOF afterward

I have an executable that starts a user-interactive shell. I would like to, upon launch of the shell, inject a few commands first, then allow the user to have their interactive session. I can do this ...
5
votes
2answers
1k views

Obtaining PID of command earlier in Pipeline

I'm writing a bash script to use inotifywait to monitor a directory and kick off actions when changes are detected. Something like: inotifywait -m ... | while read f; do something; done Since ...
2
votes
2answers
250 views

turn off buffering for `hping` in OpenBSD

It would appear that the output from hping is fully buffered when piped to perl for further line-by-line processing, so piping hping to perl doesn't work. hping --icmp-ts example.ca | perl -ne 'if ...
26
votes
7answers
30k views

how do I trim each line in bash?

I would like to remove all leading and trailing spaces and tabs from each line in an output. Is there a simple tool like trim I could pipe my output into?
2
votes
3answers
579 views

How can I tell echo/printf in Bash to swallow escape codes, based on a condition?

I have a Bash script that is being run interactively and as a cron job. When run interactively it outputs colored text on the terminal. However, when run in cron it doesn't have a terminal and ...
3
votes
4answers
271 views

saving output of another command

I need to select certain lines from the log file and also save them to a text file. I tried the following but none of them are working as expected. The file "todel.txt" shows 0 bytes. tail -f ...
2
votes
2answers
779 views

Pipe command to tail: when is the first command aborted?

In this example: $ for i in {1..3}; do sleep 1; echo $i; done | head -n 2 why is the first command (for loop) killed only just before the 3 is displayed ? I expected it to be killed right after ...
3
votes
1answer
10k views

Appending to same array in various loops, only last values remain Bash 4

I have something like this: FILES=() for i in *.map do FILES+=($i) done find /var/candy -name "chocolate_[0-9]" | while read snack do FILES+=($snack) done for file in ...
1
vote
1answer
334 views

`cat /dev/ttyACM` loses data when piped through netcat

I'm reading data from /dev/ttyACM0 using cat /dev/ttyACM0 on a Raspberry Pi. The data comes from a device, which sends lines in the form 15.00 1316013 15 where the first number is a value, the ...
2
votes
1answer
1k views

Linux nuisance: /dev/stdin doesn't work with sockets

Linux has this blitheringly annoying peculiarity that /dev/stdin won't work with sockets - it is hardcoded to return ENXIO. Try this: socat TCP-OPEN:localhost:1234 EXEC:cat\ /dev/stdin,nofork ...
2
votes
1answer
811 views

Piping Sox and FFMPEG together

I'm processing a variety of audio files in a bunch of different formats and I'd like to unify their format and configuration using FFMPEG and SoX. There are two steps to my process: Convert the ...
25
votes
6answers
4k views

Pipe to multiple files in the shell

I have an application which will produce a large amount of data which I do not wish to store onto the disk. The application mostly outputs data which I do not wish to use, but a set of useful ...
1
vote
1answer
651 views

Pipe input to another process one line at a time

I have a script which can take one file name at a time and I want to run it for many files in a folder. So, I want to pipe the output of: find . -iname '*.m' to my script but one line at a time. ...
2
votes
3answers
2k views

Running tcpdump, tee and scp

I'm trying to run the following: $ tcpdump -l -X -i eth1 -n tcp port <port> | tee capture.txt | \ scp capture.txt root@10.3.3.227:/home/checker/ tcpdump -l -X -i eth1 -n tcp port ...
13
votes
4answers
298 views

pipe, { list; } only works with some programs

Need explanations from power users for such unpredictable behaviour: ps -eF | { head -n 1;grep worker; } UID PID PPID C SZ RSS PSR STIME TTY TIME CMD root 441 2 0 ...
0
votes
1answer
1k views

What is the meaning of cut's -d and -f options

How can this pipeline be explained: cat /etc/passwd | cut -d : -f 9 2>&1 | sort | uniq 2>&1 | wc -l I'm learning Unix by myself and I don't really get it. What does cut -d : -f 7 ...
5
votes
2answers
739 views

Is there a line-by-line bash pipe?

I remember reading somewhere (then I forgot where) about a rarely used type of bash pipe that could redirect output line-by-line. In other words, rather than redirecting the output once at the end, ...
12
votes
3answers
894 views

How to display the number of lines output by a command in real time?

I'm using svn export as part of a packager script for my application, and it looks like this command, like many others, doesn't have any kind of progress bar. I have two choices at the moment: ...
1
vote
1answer
257 views

Piped Logs in Apache?

Can someone walk me though how the data flows? Need to understand how I can intercept the log data, make changes to that data in situ, then save to a file.
5
votes
3answers
3k views

Why can't certain programs (like readlink) take input from a pipe?

I have a symbolic link to a script in my $PATH whose file I wanted to edit. I forgot the filepath, so I tried to do the following: $ which my_script_link | readlink I expected that to output the ...
1
vote
1answer
213 views

Pipe not picking up stdout

I am running a java program from a OS X 10.8 bash terminal, and are trying to rederect the output it is producing. However, when either running this throug a pipe, or rederecting it to a file, the ...
4
votes
3answers
1k views

Split stdin by lines

I want to invoke a command for every line of the standard input, much like xargs, but the line should be passed as standard input, not as a command-line argument: cat some-file.txt | <magic> | ...
6
votes
6answers
1k views

Is it possible to feed Open Office via STDIN?

I have an Open Office Spreadsheet document stored inside a bash variable. I want to do something like the following to feed Open Office via STDIN: echo "$openOfficeDoc" | ooffice But it doesn't ...
1
vote
1answer
302 views

Flush data to file frequently for long running command? [duplicate]

I have a command that processes data slowly. The command processes lines from a file and writes the results to the output file data.txt: my_command > data.txt The issue I have is that I'd like to ...
1
vote
2answers
526 views

How to tell the “last” command to read from STDIN?

SERVER:~ # zcat /var/log/wtmp-20130827.gz | last -f - last: -: No such file or directory SERVER:~ # Without uncompressing the wtmp file, how can I see the output of if it with the last command?
7
votes
2answers
505 views

Use keyboard-interactive authentication when piping ssh output to other command

I'd like to pipe the output of ssh to a different command. For example: ssh myserver cat remote-file | diff local-file - The problem is that myserver asks for a password, but I can't enter it. (For ...
6
votes
1answer
5k views

find and rysnc?

I want to be able to search for files over 14 days and over 10k and than rsync those found files to a destination. Is there a way to combine these two commands? find ./ -mtime +14 -size +10k rsync ...
2
votes
1answer
93 views

Pipeline each line of a file to a specific variable in a second command

I want to execute the following instruction: basex -bword=ENTRY consulta.xq But for every line of a SampleText file (plain text mainly). The SampleText contains the following lines: hello evening ...
11
votes
1answer
9k views

Piping search term (not filename) to grep

I want to pick out certain lines containing a given number from a file. The file I want to search is called os_clusters/piRNA_clusters.bed. awk '{if (a[$0]++ == 0) {split($0,b,"."); ...
5
votes
3answers
484 views

Can you pipe to a .bash_profile function?

I received a great function for highlighting files in Apple's finder using the command-line. It's basically a wrapper for osascript. I got it from Mac OS X: How to change the color label of files ...
11
votes
2answers
9k views

How to download an archive and extract it without saving the archive to disk?

I'd like to download, and extract an archive under a given directory. Here is how I've been doing it so far: wget http://downloads.mysql.com/source/dbt2-0.37.50.3.tar.gz tar zxf dbt2-0.37.50.3.tar.gz ...
2
votes
3answers
310 views

Expand placeholders with same-named environment vars in text file?

There are a lot of variations on this question, but I wanted to ask it here anyways because I am so close (!) to finding a solution on this one - I just need a little nudge. Scenario I have several ...
11
votes
3answers
37k views

What makes a Unix process die with Broken pipe?

Here are some options I thought of, not sure which is the right one. There was an I/O error reading from the pipe. The process writing to the other end of the pipe died with a failure. All processes ...
1
vote
1answer
115 views

strange behaviour of strings command [duplicate]

I am trying to capture mysql traffic and pass those traffic to strings command as follows: tcpdump -i any -s 0 -l -w - dst port 3306 | strings This is working as expected and printing all mysql ...
1
vote
2answers
485 views

How to run chrome from command line by using process substitution?

I am using the following command without success - $ google-chrome <(grep "simple" SimpleDoc.txt) If I simply do a grep, I get - $ grep "simple" SimpleDoc.txt Very simple doc that contains ...
3
votes
2answers
4k views

Why does a named pipe not get deleted after system restart?

As far as I understand, named pipes are not written to disk but are stored in memory. Here's how I created a named pipe - $ mkfifo pipe21 $ grep "simple" SimpleDoc.txt > pipe21 & [1] 2775 $ ...
6
votes
4answers
4k views

Pipe the output of a command if it is successful

INPUT_FILE=`ls -rt $MY_DIR/FILE.*.xml | head -1 | xargs basename` I wanted to execute the second command (head -1) only if the first command is successful. How do I improve this command?
5
votes
2answers
609 views

Reusing pipe data for different commands

I would like to use the same pipe for different applications, like in: cat my_file | { cmd1 cmd2 cmd3 } Cmd1 should consume part of the input. Cmd2 should consume another part and so on. ...
3
votes
3answers
372 views

How to send part of the data through the pipe/file and later interactively?

I have a command I want to run, let's call it program. It is an interactive program similar to bash or python REPL. I want to start this program with some predefined input. I tried program << ...
1
vote
1answer
851 views

pipe curl colored output

this is the script color #!/bin/bash ESC_SEQ="\x1b[" COL_RESET=$ESC_SEQ"39;49;00m" case $1 in red ) COL_START=$ESC_SEQ"31;01m" ;; green) COL_START=$ESC_SEQ"32;01m" ;; yellow ...
2
votes
1answer
524 views

How can I work around the program failing if there is *any* stdin?

For example, the following works fine: /usr/bin/program It produces some output, and gets to result. But if I invoke it like this: echo -n | /usr/bin/program or this echo -n | bash -c ...
4
votes
1answer
2k views

How can I use tee and logger -t “some tag:”?

I've seen in another post which shows how to use tee and logger as shown below. tee >(logger) <<< "System Load is OK : $Current_loadadv" >(logger) is bash syntax to create a file ...
3
votes
5answers
700 views

How can I split a pipe into 16 byte sequences?

I'm writing a bash script to automate the decryption of a certain file; however the file is encrypted in 16 byte blocks, rather than simply encoded in its entirety. Here's some psuedo code to briefly ...
4
votes
3answers
1k views

How to capture stdout of a running process redirected to /dev/null

I want to capture stdout of running process whose stdout is being redirected to /dev/null. I tried using system call fcntl(), however the moment I close fd "1", it was being allocated for other files ...
5
votes
2answers
3k views

Problem with pipes. Pipe terminates when reader done

I am on OSX, using bash, trying to make sense of pipes. I wish to let a program communicate in two directions with a bash shell. I want to set this up in such a way that this is always the same shell, ...
3
votes
2answers
1k views

Does named pipe modify the filesystem?

As title. If I create a named pipe and then read/write on it, is the filesystem where the named pipe resides affected? I.e. is the data buffered on the filesystem until read, or does it reside in ...
0
votes
1answer
357 views

What to do when result from pipe is not last argument in next command

The example below shows what I want to do, namely take the result from my last pipe and put it into the middle of the next command (where I wrote $RESULT_FROM_FIND_COMMAND). $ find . -name ...
2
votes
1answer
2k views

Bash pipe output to more

I am wondering about some bash environment setting here: is it possible to set stdout of bash as a pipe to /bin/more? You know, like using vi as editor for your commands it should be possible to avoid ...