A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

learn more… | top users | synonyms

6
votes
1answer
126 views

Function to simplify grep with an often used log

First off, sorry if this is a painfully newbish question! I'm trying make a function that simplifies grepping a log I have to work with on a regular basis. I'd like to use extended regexp with it ...
3
votes
2answers
60 views

can not access standard output from a command

I'm using this command format: cmd0 > >(cmd1) 2> >(cmd2) Cmd2 needs to echo data back to zenity as it processes some data, but zenity never hears the echos. It seems the echos are lost ...
1
vote
2answers
31 views

How can I open the file with the most matches for a given regex?

Let's say I have a directory ~/mydir that has a whole bunch of text files in it. I want to search for searchterm in this directory and then view the file that has the most matches. How can I do this ...
4
votes
2answers
167 views

bash command that outputs the result of a previous pipe

I've got a text file outputted from a WebSpider. The Spider extracts all sentences from a given list of URLs. What I need to do is then process this file and find all lines that contain more than 65 ...
1
vote
1answer
85 views

How to simulate pipe append?

I am using curlftpfs to mount remote drive. Unfortunately, with the file located in remote side, I can't use >> operation. As I understood from this link it's not possible to append to existing ...
-3
votes
2answers
65 views

Can not pipe the `pwd` result to tee [closed]

I use Ubuntu 14.04. When i execute ls | tee - it works fine. But pwd | tee - just output absolute path only once and does not exit! i need to Ctrl+C to stop it. Any clue? EDIT: I use bash.
2
votes
0answers
111 views

Understanding inotifywait, pipes and buffers

I want to monitor every file change in a directory with inotifywait. inotifywait shall write to a FIFO buffer, which then can be read, leisurely. While experimenting with relatively huge amounts of ...
1
vote
1answer
167 views

How to force ssh-add to ask passphrase when loading RSA key from FIFO?

I've created the following RSA key (with passphrase): ssh-keygen -t rsa -f rsa_foo Now when adding the key in normal way, ssh-add asks for the passphrase fine: $ ssh-add rsa_foo Enter passphrase ...
2
votes
2answers
208 views

How to change permission of anonymous pipe?

I've the script which loads the SSH key from the variable (as part of script in CI environment) in order to not keep the private file in the public repository, however ssh-add complains about the ...
2
votes
0answers
45 views

How many ways are there to convert and pass stdin as command parameter?

I am new to Linux shell. I know that xargs can be used to accept stdin from a pipe and pass it to another command as parameter. Are there other ways to do so? Is there a list I can refer to? A non-...
1
vote
0answers
161 views

csh: redirecting output from foreach

In csh: How do I redirect the output from a foreach-loop to a pipe? This works but uses a temp file: set tmpfile=/tmp/file ...
2
votes
1answer
89 views

How can I know if a command works with pipe?

I am new to Linux shell pipe. The question may not make sense so. I saw this command: docker inspect --format '{{.State.Running}}' $(docker ps -lq) and this: docker ps -l -q|while read cid b; do ...
4
votes
2answers
333 views

Putting process substituted file descriptor into variable

I have long command roughly this: $ command-outer "long command which references a file here: filename1, and another file reference here: filename2" The files are the output of another command. So ...
5
votes
1answer
274 views

Get compressed size of piped output with gzip?

To get uncompressed size of already compressed file, I can use -l option in gzip utility: gzip -l compressedfile.gz However is there a way to get size of compressed file if I am piping the output? ...
-1
votes
4answers
185 views

How to grep AND pattern with pipe

I wrote a piece of script similar to something below: srvc_location=`ps -ef| grep '${sandbox}.*pset' | head 1 | awk {print $9}` echo "SERVICE LOC : "$srvc_location But strangely this $srvc_location ...
3
votes
1answer
136 views

less: Go to end of as much of the buffer as is loaded, not the actual end of the buffer

I'll often use less to page through the output of a program that's going to keep producing output, such as with tail -f /var/log/messages | less. Is it possible to jump to the bottom of the current ...
2
votes
2answers
1k views

pipe stdout-to-file with wc

I'm working on a lab assignment for a class, which exemplifies users | wc -w as displaying the number of users currently logged in. The instructor has asked we write the output of users to a file (...
17
votes
2answers
462 views

timeout, breaking pipes and wc

I had an idea to quickly benchmark some decompression programs. E.g. for gz, I would run the command: timeout 10 zcat foo.gz | wc -c Which would measure the amount of data the decompressor could ...
1
vote
2answers
223 views

How to redirect output of a program to a ZIP file? [closed]

I am running a script which is collecting a log of a server. I need to redirect this logs to a ZIP file. Right now I am collecting data into text file. How can I redirect it directly to ZIP?
1
vote
1answer
40 views

Array from piped commands fails

First off, apologies if this has been asked before, I searched both here and StackOverflow, tried the man pages, and I'm still drawing a blank. I'm trying to write a script that will auto-mount our ...
1
vote
2answers
81 views

How to redirect dd to pv? [duplicate]

This is my dd command which I need to modify: dd if=/tmp/nfs/image.dd of=/dev/sda bs=16k Now I would like to use pv to limit the speed of copying from NFS server. How can I achieve that? I know ...
1
vote
1answer
32 views

conditional coloring using perl expression

I have two commands in perl and one test file test.txt: $ cat test.txt aaa bla bbb foo #ccc bar ddd 444 First perl expression colorizes any occurrence of aaa or bbb with red color: cat test.txt | ...
2
votes
2answers
44 views

Pipe Find Results for '*.squashfs' to unsquashfs

I am looking for something that will find the squashed filesystem, and the pass the output to unsquash the fs and it would have to be an absolute path outside the squashfs. example /tmp/mnt/live/...
14
votes
5answers
1k views

Is this a bug in bash? `return` doesn't quit function if called from a pipe

I'm been having some weird problems with bash lately. While trying to simplify my script, I came up with this small piece of code: $ o(){ echo | while read -r; do return 0; done; echo $?;}; o 0 $ o(){...
6
votes
2answers
592 views

Send the password through stdin in `ssh-copy-id`

I need to automate some identity deployments, ideally using ssh-copy-id. I'm trying to provide the password through stdin, which is possible on ssh by using the -S flag. I'm aware that I can send ...
1
vote
0answers
64 views

UDEV-rule:Open new gnome-terminal window on usb-plugin AND activate it for stdin/stdout/stderr?

My attempt is this: If a certain USB-device is plugged in, I want to open a gnome-terminal window in my desktop-environment. This newly opened window must be able to be interactive, this means, the ...
5
votes
1answer
285 views

logging to /dev/xconsole does not work properly

I am using rsyslog, and I want to send logs to /dev/xconsole. I have following relevant line in /etc/rsyslog.conf *.*;cron.!=info;mail.!=info |/dev/xconsole This works only partially. It looks ...
2
votes
2answers
54 views

why my first pipe operation didn't remove newline character?

My nextermCensus.lis contains newline character and I am only interested in the first line, I'd like to remove newline character. but it doesn't work. so I have to use second method to remove it. $(...
0
votes
1answer
75 views

Find mails, pipe them to php and delete after

I have a script that saves to DB content of piped to it file. cat /mailpath/Maildir/cur/mailfile | php -q /scriptpath/mailPipe.php I need a shell script or oneliner to find all files in Maildir, ...
2
votes
3answers
105 views

Why does using pipe '|' terminate the second process, is there a way to prevent it?

Just to give some context, I'm trying to run this command echo "set hive.execution.engine=tez;" | hive hive terminates as soon as "set hive.execution.engine..." has been entered into the hive ...
0
votes
1answer
57 views

Fetch second and fourth column from a file

Problem Statement I am not able to get the values in second and fourth column. Tried grep -i "AUGC\|AACN\|AACY" output.csv | cut -d ',' 2,4 DATA success,1234,["madhu",{'first',[{'last',18}]},"I0"...
2
votes
3answers
38 views

Is there a way to colourize the output of a command without using bashrc/bash_profile?

I have made a program where I can read several logs from different programs, I want to colourize certain terms in the output , so I can find the more interesting parts. (I will not use grep or ...
4
votes
2answers
185 views

SSH: Provide additional “pipe” fds in addition to stdin, stdout, stderr

When connecting to a host with SSH, usually three "pipes" are provided between host and guest, for stdin, stdout, and stderr. Is there a command-line option to create forwards for additional file ...
0
votes
1answer
31 views

Having extra parameters in piping or redirection

I have two programs and I want to pipe them into each other but they take extra parameters: prog1 <inputfile> <outputfile> min max prog2 <inputfile> <outputfile> min max ...
0
votes
1answer
129 views

cat SQL files and pass to import schema

I am trying to import a series of .SQL files which map one file per DB object into mysql without calling mysql 50 times, something like: mysql -u root < cat schema/db1/*.sql This isn't working ...
26
votes
5answers
2k views

Why pipe to cat only to redirect? [duplicate]

I occasionally see things like: cat file | wc | cat > file2 Why do this? When will the results (or performance) differ (favourably) from simply: cat file | wc > file2
8
votes
4answers
403 views

How can I filter the contents of a tar file, producing another tar file in the pipe?

Consider a single tar file from an external system which contains some directories with various attributes which I want to retain such as permissions, mtimes, etc. How can I easily take a subset of ...
3
votes
1answer
188 views

How do I append an item to an array in a pipeline? [duplicate]

This script should simply add a value to an array through a loop and then show all items of an array. #!/bin/bash data_file="$1" down=() counter=0 cat $data_file | while read line; do \ isEven=...
6
votes
1answer
117 views

Extra output on input redirection

I have a folder with three files: $ ls a b c If I pipe the output of ls to wc, I get the right result: $ ls | wc -l 3 However, when I specify the input to wc as the output of ls, I get extra ...
0
votes
1answer
1k views

piping a password located in a file to the ssh command [duplicate]

So I need to login as a lot of different users to a server using ssh and I'm tired of copying the password from the files, which contain the passwords for the users, each time. I can't change it to ...
0
votes
1answer
263 views

How to use pipe to assign a variable [duplicate]

I am new to bash script. Probably this is very easy but for me it's not working. How can I assign value of $x1+$x2 to bc by piping. here in my code $bc holds no value. echo -n 'value x1 : ' read x1 ...
5
votes
3answers
795 views

Behaviour of “|” pipe in linux with “>” output redirection

I am executing command ls > a.txt | sort > b.txt This command is doing the below things : executing ls sorting it creating a.txt and storing sorted output to a.txt creating b.txt , but its ...
-2
votes
2answers
659 views

Pipe Named Fifo [closed]

I am implementing a named pipe to write and read data between two processes in cpp. The first process acquires a feature point from an image in real time while the second reads this feature point. It ...
1
vote
1answer
120 views

unzip a file and pipe to “ddrescue” command

I have a zipped image file, and need use "ddrescue" command instead "dd" to do something like this: unzip -p ubuntuXXXX.img.zip | sudo dd bs=2M of=/dev/sdb But "ddrescue" needs the image file as ...
3
votes
4answers
75 views

Who forks the processes of piped commands in the shell?

Consider this command in the shell : ls|less Now we know that ls and less are run as two different processes. We know that the shell forks and creates a child process which load the ls program using ...
0
votes
4answers
119 views

piping commands not working as expected I need explanation

I am trying to do the below touch ~/scratches/scr6.txt | cat ~/scr.txt > ~/scratches/scr6.txt | cat /dev/null > ~/scr.txt I don't know why scr6.txt is always empty, however doing the below ...
3
votes
1answer
510 views

Round-tripping text through zip, unzip fails with 'extra bytes' error on OS X [duplicate]

I'm trying to zip streaming text, and unzip it later. A minimal example is: echo "Hey, could you zip this for me?" | zip hello.zip - ; unzip -p hello.zip This fails, with the following output: ...
1
vote
1answer
188 views

Deadlock on read/wait [closed]

My process deadlocks. master looks like this: p=Popen(cmd, stdin=PIPE, stdout=PIPE) for ....: # a few million p.stdin.write(...) p.stdin.close() out = p.stdout.read() p.stdout.close() exitcode = ...
14
votes
2answers
522 views

Resource usage using pipe and here string

We can get the same result using the following two in bash, echo 'foo' | cat and cat <<< 'foo' My question is what are the difference between these two as far as the resources used are ...
1
vote
1answer
51 views

CD with redirect to logger does not work

On Linux, the following cd works fine so cds into tmp: / # cd /tmp tmp # But this does not work (stays at root, nothing appears in /var/log/messages). Why not? / # cd /tmp 2>&1 | /usr/bin/...