A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

learn more… | top users | synonyms

0
votes
1answer
15 views

Creating a function that reads off an input list of files

I am new to UNIX/Linux and am in the process of writing bioinformatics pipelines. These pipelines take in input files and pass them through multiple packages. Say there is a list of files that goes ...
2
votes
1answer
61 views

How to trick an application expecting a file:// to read from stdin?

A file URI scheme makes it possible to specify an URI to a file stored locally, such as file:///home/demo/sample. Could it be used to specify stdin? For instance, with CURL: curl ...
0
votes
0answers
19 views

Piped sequence - printing

I need to write a piped sequence of commands that prints out a list of all sub directories of a current directory. In order to get all sub directories of a current directory I use: ls -D Since I am ...
0
votes
1answer
45 views

Delete Empty Folders - recursive [duplicate]

I would like to clean up some folders. So I want to delete all folders which are empty or only contain other folders... First I tried find . -type d -delete but this lists the subfolders after ...
3
votes
3answers
512 views

How do I pipe the output of tar through mv?

A previous question of mine asked how to pipe downloaded files through tar, now I would like to know how to pipe the output of tar through mv. See I have this command at the moment: wget -c ...
1
vote
1answer
128 views

Pass a value for an input prompt

I do unzip my.zip and if files already exists it asks replace myfile.ext [y]es, [n]o, [A]ll, [N]one, [r]ename: How can I force input for when the command prompts for input? I am aware unzip may have ...
3
votes
2answers
124 views

Extract compressed file by it's header as it is piped from stdout (locally or from remote location)?

I am sending compressed file with piping either local or from a network location. And on the receiving end, I would like to detect the type of compression and use the appropriate decompression utility ...
1
vote
0answers
56 views

Chaining grep commands [duplicate]

I'd like to find all xml files containing 'foo', but not containing 'bar'. The command I've put together looks like this: find -iname "*.xml" -print0 | xargs -0 grep -FlisZ "foo" | xargs -0 grep ...
1
vote
1answer
116 views

Extract characters from a serial port

I have a dummy question for you, I am trying to extract some lines from a file beginning by a special string of characters and to copy them into a file. I tried : cat /dev/ttyACM0 | grep "something" ...
1
vote
2answers
164 views

How do I pipe the files downloaded with wget through tar?

I would like to pipe tarballs (.tar.gz files in this case) I download with GNU Wget (wget) through tar -xzf (to decompress them, if this is unclear) but I don't know how. I have tried: wget -q -O- ...
4
votes
1answer
72 views

Is this tail behavior in Grouping Commands specified by POSIX?

Using tail combined with other standard tools in Grouping Commands can make some powerful constructions. For example, to get the first and last line of a file: $ seq 10 > file $ { head -n1; tail ...
0
votes
1answer
103 views

Grep -v filter not working

I want to filter grep results by using grep -v option. But the output does not differ when using a particular pattern. contents of log.log: ERROR error EXCEPTION exception ...
2
votes
5answers
200 views

How do I cat a file I got from stdin without xargs

In an effort to learn the shell better, and without always having to resort to xargs, I have been trying to discover any other ways to do: find . -name *.tcl | xargs -I{} cat {} xargs make it feel ...
2
votes
2answers
52 views

Insert result of command into an executable (one command) [duplicate]

Is it possible to insert the result of a command (or even a chain of commands) into an executable in one command? Something like this: ./a.out < echo 0 Or is it necessary to do this: echo 0 ...
0
votes
1answer
63 views

Pretty tricky cmd [duplicate]

I was searching for a cmd to generate random passwords, and got stuck on this one : echo `< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c6` I've read the documentation of urandom, tr and head. But ...
2
votes
2answers
33 views

All lines not moving through pipe?

With slabtop I get the following output (50 lines): $slabtop -sc -o Active / Total Objects (% used) : 110864927 / 111473562 (99.5%) Active / Total Slabs (% used) : 2826375 / 2826375 ...
6
votes
1answer
124 views

Function to simplify grep with an often used log

First off, sorry if this is a painfully newbish question! I'm trying make a function that simplifies grepping a log I have to work with on a regular basis. I'd like to use extended regexp with it ...
3
votes
2answers
58 views

can not access standard output from a command

I'm using this command format: cmd0 > >(cmd1) 2> >(cmd2) Cmd2 needs to echo data back to zenity as it processes some data, but zenity never hears the echos. It seems the echos are lost ...
1
vote
2answers
28 views

How can I open the file with the most matches for a given regex?

Let's say I have a directory ~/mydir that has a whole bunch of text files in it. I want to search for searchterm in this directory and then view the file that has the most matches. How can I do this ...
3
votes
2answers
160 views

bash command that outputs the result of a previous pipe

I've got a text file outputted from a WebSpider. The Spider extracts all sentences from a given list of URLs. What I need to do is then process this file and find all lines that contain more than 65 ...
1
vote
1answer
75 views

How to simulate pipe append?

I am using curlftpfs to mount remote drive. Unfortunately, with the file located in remote side, I can't use >> operation. As I understood from this link it's not possible to append to existing ...
-3
votes
2answers
59 views

Can not pipe the `pwd` result to tee [closed]

I use Ubuntu 14.04. When i execute ls | tee - it works fine. But pwd | tee - just output absolute path only once and does not exit! i need to Ctrl+C to stop it. Any clue? EDIT: I use bash.
2
votes
0answers
85 views

Understanding inotifywait, pipes and buffers

I want to monitor every file change in a directory with inotifywait. inotifywait shall write to a FIFO buffer, which then can be read, leisurely. While experimenting with relatively huge amounts of ...
1
vote
1answer
127 views

How to force ssh-add to ask passphrase when loading RSA key from FIFO?

I've created the following RSA key (with passphrase): ssh-keygen -t rsa -f rsa_foo Now when adding the key in normal way, ssh-add asks for the passphrase fine: $ ssh-add rsa_foo Enter passphrase ...
2
votes
2answers
172 views

How to change permission of anonymous pipe?

I've the script which loads the SSH key from the variable (as part of script in CI environment) in order to not keep the private file in the public repository, however ssh-add complains about the ...
2
votes
0answers
41 views

How many ways are there to convert and pass stdin as command parameter?

I am new to Linux shell. I know that xargs can be used to accept stdin from a pipe and pass it to another command as parameter. Are there other ways to do so? Is there a list I can refer to? A ...
1
vote
0answers
116 views

csh: redirecting output from foreach

In csh: How do I redirect the output from a foreach-loop to a pipe? This works but uses a temp file: set tmpfile=/tmp/file ...
2
votes
1answer
68 views

How can I know if a command works with pipe?

I am new to Linux shell pipe. The question may not make sense so. I saw this command: docker inspect --format '{{.State.Running}}' $(docker ps -lq) and this: docker ps -l -q|while read cid b; do ...
4
votes
2answers
328 views

Putting process substituted file descriptor into variable

I have long command roughly this: $ command-outer "long command which references a file here: filename1, and another file reference here: filename2" The files are the output of another command. So ...
5
votes
1answer
177 views

Get compressed size of piped output with gzip?

To get uncompressed size of already compressed file, I can use -l option in gzip utility: gzip -l compressedfile.gz However is there a way to get size of compressed file if I am piping the output? ...
-1
votes
4answers
146 views

How to grep AND pattern with pipe

I wrote a piece of script similar to something below: srvc_location=`ps -ef| grep '${sandbox}.*pset' | head 1 | awk {print $9}` echo "SERVICE LOC : "$srvc_location But strangely this $srvc_location ...
3
votes
1answer
104 views

less: Go to end of as much of the buffer as is loaded, not the actual end of the buffer

I'll often use less to page through the output of a program that's going to keep producing output, such as with tail -f /var/log/messages | less. Is it possible to jump to the bottom of the current ...
2
votes
2answers
794 views

pipe stdout-to-file with wc

I'm working on a lab assignment for a class, which exemplifies users | wc -w as displaying the number of users currently logged in. The instructor has asked we write the output of users to a file ...
17
votes
2answers
452 views

timeout, breaking pipes and wc

I had an idea to quickly benchmark some decompression programs. E.g. for gz, I would run the command: timeout 10 zcat foo.gz | wc -c Which would measure the amount of data the decompressor could ...
1
vote
2answers
145 views

How to redirect output of a program to a ZIP file? [closed]

I am running a script which is collecting a log of a server. I need to redirect this logs to a ZIP file. Right now I am collecting data into text file. How can I redirect it directly to ZIP?
1
vote
1answer
39 views

Array from piped commands fails

First off, apologies if this has been asked before, I searched both here and StackOverflow, tried the man pages, and I'm still drawing a blank. I'm trying to write a script that will auto-mount our ...
1
vote
2answers
72 views

How to redirect dd to pv? [duplicate]

This is my dd command which I need to modify: dd if=/tmp/nfs/image.dd of=/dev/sda bs=16k Now I would like to use pv to limit the speed of copying from NFS server. How can I achieve that? I know ...
1
vote
1answer
32 views

conditional coloring using perl expression

I have two commands in perl and one test file test.txt: $ cat test.txt aaa bla bbb foo #ccc bar ddd 444 First perl expression colorizes any occurrence of aaa or bbb with red color: cat test.txt | ...
2
votes
2answers
35 views

Pipe Find Results for '*.squashfs' to unsquashfs

I am looking for something that will find the squashed filesystem, and the pass the output to unsquash the fs and it would have to be an absolute path outside the squashfs. example ...
14
votes
5answers
1k views

Is this a bug in bash? `return` doesn't quit function if called from a pipe

I'm been having some weird problems with bash lately. While trying to simplify my script, I came up with this small piece of code: $ o(){ echo | while read -r; do return 0; done; echo $?;}; o 0 $ ...
6
votes
2answers
393 views

Send the password through stdin in `ssh-copy-id`

I need to automate some identity deployments, ideally using ssh-copy-id. I'm trying to provide the password through stdin, which is possible on ssh by using the -S flag. I'm aware that I can send ...
1
vote
0answers
47 views

UDEV-rule:Open new gnome-terminal window on usb-plugin AND activate it for stdin/stdout/stderr?

My attempt is this: If a certain USB-device is plugged in, I want to open a gnome-terminal window in my desktop-environment. This newly opened window must be able to be interactive, this means, the ...
5
votes
1answer
217 views

logging to /dev/xconsole does not work properly

I am using rsyslog, and I want to send logs to /dev/xconsole. I have following relevant line in /etc/rsyslog.conf *.*;cron.!=info;mail.!=info |/dev/xconsole This works only partially. It looks ...
2
votes
2answers
52 views

why my first pipe operation didn't remove newline character?

My nextermCensus.lis contains newline character and I am only interested in the first line, I'd like to remove newline character. but it doesn't work. so I have to use second method to remove it. ...
0
votes
1answer
71 views

Find mails, pipe them to php and delete after

I have a script that saves to DB content of piped to it file. cat /mailpath/Maildir/cur/mailfile | php -q /scriptpath/mailPipe.php I need a shell script or oneliner to find all files in Maildir, ...
2
votes
3answers
103 views

Why does using pipe '|' terminate the second process, is there a way to prevent it?

Just to give some context, I'm trying to run this command echo "set hive.execution.engine=tez;" | hive hive terminates as soon as "set hive.execution.engine..." has been entered into the hive ...
0
votes
1answer
57 views

Fetch second and fourth column from a file

Problem Statement I am not able to get the values in second and fourth column. Tried grep -i "AUGC\|AACN\|AACY" output.csv | cut -d ',' 2,4 DATA ...
2
votes
3answers
37 views

Is there a way to colourize the output of a command without using bashrc/bash_profile?

I have made a program where I can read several logs from different programs, I want to colourize certain terms in the output , so I can find the more interesting parts. (I will not use grep or ...
4
votes
2answers
144 views

SSH: Provide additional “pipe” fds in addition to stdin, stdout, stderr

When connecting to a host with SSH, usually three "pipes" are provided between host and guest, for stdin, stdout, and stderr. Is there a command-line option to create forwards for additional file ...
0
votes
1answer
30 views

Having extra parameters in piping or redirection

I have two programs and I want to pipe them into each other but they take extra parameters: prog1 <inputfile> <outputfile> min max prog2 <inputfile> <outputfile> min max ...