Chains the standard streams of a series of commands

learn more… | top users | synonyms

33
votes
3answers
3k views

In what order do piped commands run?

I've never really thought about how the shell actually executes piped commands. I've always been told that the "stdout of one program gets piped into the stdin of another," as a way of thinking about ...
2
votes
1answer
92 views

pipe and log STDOUT at the same time

Some programs output very detailed data, for readability I use awk to filter them. ./output_many_things | awk '{print $1 "\t" $2}' But I don't want to miss anything. Could I redirect the complete ...
2
votes
1answer
739 views

Using Putty/plink to scp to a named pipe

I have a large tar.gz file that I need to transfer from Windows to a UNIX machine and unzip into a folder that my login user doesn't have write permissions on. Once the file is on the machine I can ...
4
votes
1answer
245 views

command like wc but with tee behaviour

I want to backup a database using psql and COPY command. here is my script: psql "user=${USERNAME} host=${HOSTNAME} dbname=${DBNAME} password=${PASSWORD}" -c \ "COPY (SELECT * FROM tbl) ORDER BY id ...
7
votes
4answers
696 views

What is the easiest way to execute text from tail at the command line?

Sometimes I'm working on a new (ubuntu) box and I type git and am alerted: The program 'git' is currently not installed. You can install it by typing: apt-get install git-core If that happens I ...
3
votes
1answer
1k views

Command line audio - piping for simultaneous playback and recording

I am trying to generate sound data, convert it and store it to a WAV format. I'm almost there - except I'd like to hear the generated sound "while" it is being "recorded". This command line just ...
3
votes
1answer
159 views

How can I pipe a path to pushd?

This seems like it should be easy enough to do, but I'm clearly not understanding something fundamental about piping output back and forth. I'm trying to do something like this: bundle show ...
5
votes
2answers
2k views

Running piped bash script in background

I'm attempting to build a monitoring script to watch localhost communication using netcat. I have two scripts that I've built, one to start the monitoring loop and one for the loop itself. They are as ...
2
votes
2answers
2k views

Using a named pipe as ffmpeg/avconv's input for streaming

So I'm in a situation in which I want to feed a .wav (or other kind of sound) file to a named pipe/FIFO, and use the pipe as ffmpeg's input source (with a ffserver feed as an output). This is as far ...
3
votes
3answers
2k views

How can I pass output of one command as an argument to another

A similar question has been asked, but since I am new to Unix the answer was not clear to me due to the context. What I want to do is to pass the output of one command as an argument to another. I am ...
26
votes
4answers
6k views

Can I pipe stdout on one server to stdin on another server?

stdout on one CentOS server needs to be piped to stdin on another CentOS server. Is this possible? Update ScottPack, MikeyB and jofel all have valid answers. I awarded the answer to Scott because, ...
3
votes
1answer
134 views

Why does tailing an output log sometimes give partial lines?

Sometimes, tailing an output log which is constantly being updated doesn't give the whole lines. Why is that? grep pattern input_file > output.log & tail output.log Why doesn't it print the ...
6
votes
2answers
5k views

gzip - redirection or piping?

I will be backing up a large (750GB) disk to an external USB disk using dd. Should I be using redirection or piping? Which is more efficient? Or is there a difference? Also, what is the best block ...
6
votes
4answers
257 views

How can I print a conditional header BEFORE stdout, if there is any output on stdout

I have a process that filters a list of files from a directory (having find check to see if there are files older than a certain period to show a queue is stuck). It may or may not return anything, ...
0
votes
2answers
460 views

ls outputs multiple columns even if piped to grep with grml's zsh config

I just decided to give zsh a go against Bash and came across some unhandy behaviour about which I couldn't find anything on the net: If you do a ls | grep foo in Bash, the ls only contains one file ...
5
votes
1answer
657 views

TLS over unix pipe

Can I use TLS/SSL over Unix pipe with Unix command line? I want the equivalent of $ mkfifo /tmp/spipe $ echo a|openssl s_server -acceptFifo /tmp/spipe & [1] 25563 $ openssl s_client ...
6
votes
1answer
4k views

How to check if pipe is empty

I have piped line in bash script and want to check if pipe has data, before feeding program Searching I found about test -t 0 but it doesn't work here. Always returns false. So how to be sure that ...
8
votes
3answers
2k views

Have Bash script wait for status message before continuing

I'm firing up Selenium server with a bash script and as you can see from the timestamps on the log below, it takes about 32 seconds for the thing to fully come online: Feb 28, 2012 10:19:02 PM ...
1
vote
3answers
3k views

grep piping into sed, replacing inline; but I want sed to print the filename and changed line. Is it possible?

Here's my command (break intentional): grep FOO "/Users/gjtorikian/blah" -l | xargs sed -i '' '/FOO/{s/FOO/BAR/g; w /dev/stdout }' At the high-level: grep for FOO in the blah directory; pipe in ...
4
votes
3answers
293 views

How to pipe to multiple places with vertical bar?

I have a folder full of files with a .dot extension like this: a.dot b.dot c.dot I want to go through all the files in this folder and run a command on them such as this: dot -Tpdf a.dot -o a.pdf ...
2
votes
1answer
649 views

Keep global variables values piping through functions

I wrote a little bash script using sed on some html pages to extract some urls. To avoid each time grabbing sed results in a variable then read it again I simply made 3 functions and piped together. ...
0
votes
1answer
691 views

Egrep: contains and does not contain

egrep -i '12-12.*\swarning' log.file | cless The above will produce two lines with the date 12-12, followed by random characters, a white space, then Warning and more text. 2011-12-12 textetc ...
5
votes
1answer
146 views

Confusion about sed and '>'

The following command works fine: sed s/input/raw_input/ p.py >p2.py && mv p2.py p.py However, the following command turns p.py into an empty file: sed s/input/raw_input/ p.py >p.py ...
4
votes
2answers
846 views

Using files that have spaces in their name in pipes

I have folder named play which contains a file damn file (with space). When I use find and pipe the output to tar: find play/ -name 'damn*' | tar cf archive.tar -T - It works perfectly. Why does ...
6
votes
2answers
3k views

what is meant by connecting STDOUT and STDIN?

This is a newbie question. I'm reading a book, it says: Every process has at least three communication channels available to it: “standard input” (STDIN), “standard output” (STDOUT), and ...
4
votes
2answers
275 views

How to expand tabs based on content?

I've got some tab-delimited data coming out of a Unix pipe. I'd like to format this data into a compact human-readable table. How can I expand these tabs into spaces, and automatically set the tab ...
5
votes
2answers
140 views

What is “blocking” in reference to file I/O?

My textbook mentions it but doesn't specifically define it. From the context it's used in (in this example, pipes), I take it to mean that if one end of the pipe blocks, it's temporarily "disabled", ...
3
votes
2answers
725 views

Is it possible to make seek() operations on a named pipe return successful?

Is there any way to make it so that when programs try to perform seek() operations on a named pipe it will come back successful (but act as if the pipe was an empty file) instead of 'Illegal seek'? I ...
15
votes
10answers
7k views

What's a good example of piping commands together?

If you were helping someone to learn the concept of pipes on the command line what example would you use? The example that actually came up was as follows: cat whatever.txt | less I feel like ...
2
votes
2answers
818 views

How can I retain the console input in mplayer when reading from stdin?

I'm playing around with the command line interface of mplayer. I'd like to script it in the following way find /some/path/ -type f | grep -vif blacklist | mplayer -shuffle -playlist - where ...
20
votes
4answers
726 views

Are Linux utilities smart when running piped commands?

I was just running a few commands in a terminal and I started wondering, does Unix/Linux take shortcuts when running piped commands? For example, let's say I have a file with one million lines, the ...
7
votes
5answers
3k views

Shell Script mktemp, what's the best method to create temporary named pipe?

I'm aware its best to create temporary files with mktemp, but what about named pipes? I have an idea, I'm sure it would work, but I would really like an expert opinion. I prefer things to be as ...
3
votes
1answer
442 views

ssh “-f” don't give back hand in bash when listening to stdout/stderr

Please look at these: ## Does NOT return to the shell, but Ctrl-C can exit ssh -S none -fNR 13018:localhost:22 example.com | cat ## Returns to the shell (no "-S none") ssh -fNR ...
4
votes
1answer
213 views

Why did using xargs fail in this case?

I tried to understand the usage of xargs and did the following experiment. ls | xargs | touch I want to refresh the files dates and directoris in the curent directory. Though it is a bit silly,for ...
1
vote
1answer
251 views

How to know if a Linux command can read from STDIN?

I noticed this problem when I became confused with pipe, one command send its executing output to the STDOUT, which is the STDIN for the other command, which can read from STDIN. How do I know if a ...
4
votes
2answers
1k views

Follow a pipe using less?

Can less follow (by pressing F) a piped input (similarly to a file)? For a file that is being written to, the command less <file> will follow the file when pressing F. But if I have a ...
9
votes
1answer
816 views

Readable comments on separate lines in a multi-line bash command with pipelines?

When creating shell scripts using pipelines, and using the backslash to continue lines, I want to insert comments on separate lines, in a robust, readable and portable fashion. For example, given ...
2
votes
1answer
2k views

How to set a new password from bash?

I'm using busybox with a limited passwd (I don't have --stdin option) and without chpasswd and I need to change the password of an user from bash. Here is my best result: echo newpassword > ...
2
votes
2answers
985 views

Passing values through pipes

I don't quite understand pipes in Linux command line. I noticed that: ll - R | grep *.pdf will list files ending with .pdf But locate *.pdf | du -h will not calculate the size of files ...
11
votes
3answers
2k views

How can I pipe output to another process, but retain the error state of the first process? [duplicate]

Possible Duplicate: Get exit code of process that's piped to another I am using the following command line (in a makefile) to pipe the verbose error messages from my compiler through a ...
3
votes
2answers
1k views

How can I pipe input from one SSH session to another? Using fifo?

A while ago I saw an article about piping messages from one console window to another using fifo ? I haven't used that before and can't find my link to the article. How can I set this up? I'd like ...
7
votes
3answers
1k views

Piping STDERR vs. STDOUT

According to "Linux: The Complete Reference 6th Edition" (pg. 44), you can pipe only STDERR using the |& redirection symbols. I've written a pretty simple script to test this: #!/bin/bash echo ...
2
votes
2answers
852 views

I need to transfer a very large folder over FTP, without creating a file first

I am running an Amazon EC2 instance that is limited in disk space. I have a very large folder that I need to FTP to my local machine, preferably without creating an intermediate tar file, since there ...
9
votes
3answers
4k views

Non-blocking buffered named pipe?

I'm looking for something I suspect doesn't exist: A non-blocking buffered named pipe (fifo) for use from the command line. Is there such a thing? Here's the use case: Suppose I have a process that's ...
1
vote
2answers
267 views

Burning a directory structure from a stdin pipe

I'm trying to do something tricky, I want to burn a directory structure onto a CD from a pipe stream. The reason is that it is coming from the network and I don't want it written on the hard drive of ...
5
votes
1answer
361 views

Pipes vs process substitution

While trying the cat "$@" trick to read from either standard input or given files, it turned out that pipe and process substitution handle a missing trailing newline differently: printf %s foo > ...
2
votes
4answers
784 views

Open file found with 'find' command

This is probably an easy one, but I can't figure it out and it's pretty much not searchable. In a folder hierarchy I have exactly one file of type xyz. I want to find that file and open it with a ...
3
votes
3answers
1k views

Create a temporary file from a stdout redirect or pipe

Some commands only output to stdout. Some tools only work on files. Is there a command that can glue those together? Contrived simple example: diff $(echo 1 | stdout-to-temp-file) $(echo 2 | ...
2
votes
2answers
829 views

Difference between > and | with /dev/tty

When I run this command: echo "1" > /dev/tty # runs successfully but when I run this: echo "1" | /dev/tty #permission denied What is the difference between these two operators > and | and ...
4
votes
4answers
255 views

How to send stderr to stdout with a pipe to another command?

I'm trying to capture all of the output of a build operation, and send it to myprogram. I seem to only be able to capture part of it. This is what I'm trying: make clean && make ...