A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

learn more… | top users | synonyms

2
votes
1answer
95 views

pipe the output of cat into a node script

I am an absolute beginner at UNIX scripting (and have searched here for something explaining how to do this in a simple way to no avail). I am trying to pipe the contents of a claimName.txt file find ...
0
votes
1answer
45 views

Trying to Use awk by implementating a condition and implication

I have a file containing 8 columns, One of my tasks is to get the values of Column 3 and if the value of the row is not equal to 123 the 8 coloumns will be printed and hence If column 3 is equal to ...
0
votes
1answer
19 views

Getting handle on item in a pipe and piping result

I have this pretty cool command that giving me a nice json representation of a nix package nix-env --json -qaA nixpkgs.wireshark | json_pp | less But what I really want in pseudo shell/OCaml ...
0
votes
2answers
54 views

Conditional grepping

I have a scenario to handle. Need help with that... I have a config file and its content looks something like below: Jobname|Type|Silo|Description #comment1 #comment2 ...
3
votes
1answer
66 views

Rsync contents of tar with remote server

I've got a number of backups as tar archives, and to do checking/testing of them I restore to some test machines using: cat archive.tar.gz | ssh server2 "cd /path/to/dir; tar zxvf -" Most of the ...
2
votes
1answer
41 views

who to get a count of “base subDir” without there subDirs?

All I am trying to do is get a count of direcrories on one leve of a tree. so Far I have this. find /media/data/temp1 -type d -exec basename {} \; which gives me this: temp1 Coldplay Greatest ...
2
votes
2answers
131 views

Avoid output buffering for a command group (curly braces) in bash script

I have a bash script containing a group of commands in curly braces { ... }. This group contains some initial echo commands and then one loop. At each iteration the loop executes various slow commands ...
2
votes
2answers
543 views

How to download an RPM package and install it in one line?

I was wondering whether using wget it was possible to download an RPM and then pipe it through sudo rpm -i to install it, in a single line? I realize I could just run: wget -c <URL> sudo rpm -i ...
2
votes
2answers
299 views

Curl url txt file, but grep each url seperately from single file

Ok, here's one for you guys that is puzzling me. So I have a text file with lots of url's in it. I'm using curl -K "$urls" > $output to spit the output to my output file. Now for the output of ...
1
vote
3answers
51 views

Pipes vs variable assignment

In what ways are pipes different from variable assignments? I know you can't use null bytes. What other examples are there? What about for piping in something like the following yes | cat? I notice ...
1
vote
2answers
123 views

pipes - Inserting pipe to middle of string [duplicate]

How would I make a pipe insert characters into the middle of a string? Here's my example (running Raspian): hostname -I | sudo python ~/testsite/manage.py runserver {the ip from the first command}:80 ...
1
vote
1answer
67 views

How are piped processes presented in ps output?

I have a following very simple script which watches /tmp directory for new files and output of inotifywait is piped to sed: #!/bin/bash /usr/local/bin/inotifywait -q -m /tmp --format %f | sed ...
5
votes
1answer
219 views

Sort output of awk except for first line?

This is a use case I am continually running into in parsing CSV files. When it is an inline awk script embedded in a shell script, I can use this workaround: myfile=$(mktemp) awk '(awk script here)' ...
14
votes
2answers
2k views

Why does using `yes` on bash pipelines *not* cause infinite loops?

According to its documentation, bash waits until all commands in a pipeline have finished running before continuing The shell waits for all commands in the pipeline to terminate before returning a ...
2
votes
2answers
144 views

pipe ls and awk to rsync

Trying to complete a script with the following steps: (1) select 1gig of most recently changed files across a number of subdirectories (2) preferably uses rsync to copy the files to local directory -- ...
0
votes
1answer
15 views

Creating a function that reads off an input list of files

I am new to UNIX/Linux and am in the process of writing bioinformatics pipelines. These pipelines take in input files and pass them through multiple packages. Say there is a list of files that goes ...
2
votes
1answer
64 views

How to trick an application expecting a file:// to read from stdin?

A file URI scheme makes it possible to specify an URI to a file stored locally, such as file:///home/demo/sample. Could it be used to specify stdin? For instance, with CURL: curl ...
0
votes
0answers
20 views

Piped sequence - printing

I need to write a piped sequence of commands that prints out a list of all sub directories of a current directory. In order to get all sub directories of a current directory I use: ls -D Since I am ...
0
votes
1answer
46 views

Delete Empty Folders - recursive [duplicate]

I would like to clean up some folders. So I want to delete all folders which are empty or only contain other folders... First I tried find . -type d -delete but this lists the subfolders after ...
3
votes
3answers
547 views

How do I pipe the output of tar through mv?

A previous question of mine asked how to pipe downloaded files through tar, now I would like to know how to pipe the output of tar through mv. See I have this command at the moment: wget -c ...
1
vote
1answer
188 views

Pass a value for an input prompt

I do unzip my.zip and if files already exists it asks replace myfile.ext [y]es, [n]o, [A]ll, [N]one, [r]ename: How can I force input for when the command prompts for input? I am aware unzip may have ...
3
votes
2answers
132 views

Extract compressed file by it's header as it is piped from stdout (locally or from remote location)?

I am sending compressed file with piping either local or from a network location. And on the receiving end, I would like to detect the type of compression and use the appropriate decompression utility ...
1
vote
0answers
56 views

Chaining grep commands [duplicate]

I'd like to find all xml files containing 'foo', but not containing 'bar'. The command I've put together looks like this: find -iname "*.xml" -print0 | xargs -0 grep -FlisZ "foo" | xargs -0 grep ...
1
vote
1answer
145 views

Extract characters from a serial port

I have a dummy question for you, I am trying to extract some lines from a file beginning by a special string of characters and to copy them into a file. I tried : cat /dev/ttyACM0 | grep "something" ...
1
vote
2answers
178 views

How do I pipe the files downloaded with wget through tar?

I would like to pipe tarballs (.tar.gz files in this case) I download with GNU Wget (wget) through tar -xzf (to decompress them, if this is unclear) but I don't know how. I have tried: wget -q -O- ...
4
votes
1answer
72 views

Is this tail behavior in Grouping Commands specified by POSIX?

Using tail combined with other standard tools in Grouping Commands can make some powerful constructions. For example, to get the first and last line of a file: $ seq 10 > file $ { head -n1; tail ...
0
votes
1answer
106 views

Grep -v filter not working

I want to filter grep results by using grep -v option. But the output does not differ when using a particular pattern. contents of log.log: ERROR error EXCEPTION exception ...
2
votes
5answers
209 views

How do I cat a file I got from stdin without xargs

In an effort to learn the shell better, and without always having to resort to xargs, I have been trying to discover any other ways to do: find . -name *.tcl | xargs -I{} cat {} xargs make it feel ...
2
votes
2answers
52 views

Insert result of command into an executable (one command) [duplicate]

Is it possible to insert the result of a command (or even a chain of commands) into an executable in one command? Something like this: ./a.out < echo 0 Or is it necessary to do this: echo 0 ...
0
votes
1answer
66 views

Pretty tricky cmd [duplicate]

I was searching for a cmd to generate random passwords, and got stuck on this one : echo `< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c6` I've read the documentation of urandom, tr and head. But ...
2
votes
2answers
33 views

All lines not moving through pipe?

With slabtop I get the following output (50 lines): $slabtop -sc -o Active / Total Objects (% used) : 110864927 / 111473562 (99.5%) Active / Total Slabs (% used) : 2826375 / 2826375 ...
6
votes
1answer
125 views

Function to simplify grep with an often used log

First off, sorry if this is a painfully newbish question! I'm trying make a function that simplifies grepping a log I have to work with on a regular basis. I'd like to use extended regexp with it ...
3
votes
2answers
59 views

can not access standard output from a command

I'm using this command format: cmd0 > >(cmd1) 2> >(cmd2) Cmd2 needs to echo data back to zenity as it processes some data, but zenity never hears the echos. It seems the echos are lost ...
1
vote
2answers
29 views

How can I open the file with the most matches for a given regex?

Let's say I have a directory ~/mydir that has a whole bunch of text files in it. I want to search for searchterm in this directory and then view the file that has the most matches. How can I do this ...
3
votes
2answers
160 views

bash command that outputs the result of a previous pipe

I've got a text file outputted from a WebSpider. The Spider extracts all sentences from a given list of URLs. What I need to do is then process this file and find all lines that contain more than 65 ...
1
vote
1answer
79 views

How to simulate pipe append?

I am using curlftpfs to mount remote drive. Unfortunately, with the file located in remote side, I can't use >> operation. As I understood from this link it's not possible to append to existing ...
-3
votes
2answers
64 views

Can not pipe the `pwd` result to tee [closed]

I use Ubuntu 14.04. When i execute ls | tee - it works fine. But pwd | tee - just output absolute path only once and does not exit! i need to Ctrl+C to stop it. Any clue? EDIT: I use bash.
2
votes
0answers
94 views

Understanding inotifywait, pipes and buffers

I want to monitor every file change in a directory with inotifywait. inotifywait shall write to a FIFO buffer, which then can be read, leisurely. While experimenting with relatively huge amounts of ...
1
vote
1answer
142 views

How to force ssh-add to ask passphrase when loading RSA key from FIFO?

I've created the following RSA key (with passphrase): ssh-keygen -t rsa -f rsa_foo Now when adding the key in normal way, ssh-add asks for the passphrase fine: $ ssh-add rsa_foo Enter passphrase ...
2
votes
2answers
192 views

How to change permission of anonymous pipe?

I've the script which loads the SSH key from the variable (as part of script in CI environment) in order to not keep the private file in the public repository, however ssh-add complains about the ...
2
votes
0answers
43 views

How many ways are there to convert and pass stdin as command parameter?

I am new to Linux shell. I know that xargs can be used to accept stdin from a pipe and pass it to another command as parameter. Are there other ways to do so? Is there a list I can refer to? A ...
1
vote
0answers
140 views

csh: redirecting output from foreach

In csh: How do I redirect the output from a foreach-loop to a pipe? This works but uses a temp file: set tmpfile=/tmp/file ...
2
votes
1answer
75 views

How can I know if a command works with pipe?

I am new to Linux shell pipe. The question may not make sense so. I saw this command: docker inspect --format '{{.State.Running}}' $(docker ps -lq) and this: docker ps -l -q|while read cid b; do ...
4
votes
2answers
331 views

Putting process substituted file descriptor into variable

I have long command roughly this: $ command-outer "long command which references a file here: filename1, and another file reference here: filename2" The files are the output of another command. So ...
5
votes
1answer
226 views

Get compressed size of piped output with gzip?

To get uncompressed size of already compressed file, I can use -l option in gzip utility: gzip -l compressedfile.gz However is there a way to get size of compressed file if I am piping the output? ...
-1
votes
4answers
161 views

How to grep AND pattern with pipe

I wrote a piece of script similar to something below: srvc_location=`ps -ef| grep '${sandbox}.*pset' | head 1 | awk {print $9}` echo "SERVICE LOC : "$srvc_location But strangely this $srvc_location ...
3
votes
1answer
112 views

less: Go to end of as much of the buffer as is loaded, not the actual end of the buffer

I'll often use less to page through the output of a program that's going to keep producing output, such as with tail -f /var/log/messages | less. Is it possible to jump to the bottom of the current ...
2
votes
2answers
972 views

pipe stdout-to-file with wc

I'm working on a lab assignment for a class, which exemplifies users | wc -w as displaying the number of users currently logged in. The instructor has asked we write the output of users to a file ...
17
votes
2answers
459 views

timeout, breaking pipes and wc

I had an idea to quickly benchmark some decompression programs. E.g. for gz, I would run the command: timeout 10 zcat foo.gz | wc -c Which would measure the amount of data the decompressor could ...
1
vote
2answers
175 views

How to redirect output of a program to a ZIP file? [closed]

I am running a script which is collecting a log of a server. I need to redirect this logs to a ZIP file. Right now I am collecting data into text file. How can I redirect it directly to ZIP?