A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

learn more… | top users | synonyms

1
vote
2answers
21 views

Find unique lines based on only a section

I am wondering if there is a way to split lines by some character (E.g. :) and find unique lines based on one part of this split. For example if I had the following piped in via stdin file1: line 4 ...
1
vote
1answer
51 views

Broken pipe error when reading stops

I'm trying to make a quick & dirty bash function to compare the audio content of two FLAC files without decoding them into a temp file. With the flac command line tool I'm decoding each file into ...
2
votes
2answers
77 views

How to sort data stream by 2 column names (column number may vary)?

I get a stream of data from an API with which look like this: redID blueID whiteID 1 22 2 44 15 41 2 15 15 31 2 14 What I need to do, is to sort ...
1
vote
1answer
82 views

How to provoke a broken pipe

I try to debug a python script that interfaces MPD using python-mpd2. When the script tries to perform an action after being idle for a couple of hours, it terminates with an exception (...
1
vote
1answer
249 views

Get absolute path of file OS X

I am creating a script which will prompt the user to enter a location of file and it will convert it to an absolute path. This is what I have so far. I'm not sure how to append these together. ...
1
vote
1answer
37 views

Appending to a file suffix range with echo and tee

I am unable to figure out how to properly pipe and tee to append to a range of files. Instead of manually typing them out echo "help!" | tee >> file1 file2 file3 I am trying to figure out: ...
4
votes
1answer
120 views

Ctrl-c kills shell when pipeline is used in input redirection subshell

If I run a command like: cat <(echo 1 | pv) | pv cat <(echo 1 | pv) | less cat <(echo 1 | pv) | cat The command seems to run forever. Entering ^C (SIGINT) kills the entire shell instead ...
1
vote
2answers
106 views

cryptsetup: Attaching loopback device failed

I am using cryptsetup with detached header. For that I am using the --header parameter, to specify header file. I can open my device using: cryptsetup luksOpen /dev/sdb1 sdb1 --header LUKS-HEADER ...
1
vote
1answer
58 views

Unix awk decimal export in pipe

I'm struggling to adapt a script, which loops through a large number of individual files (subj_id), and prints out the value of $4 in a specified line. Text file contains lines # Measure Brain, ICV, ...
2
votes
1answer
64 views

gnuplot output of a pipe

Inspired by this question I tried to pipe output to gnuplot and then plot it. My pipe is: cat file.txt | grep CU | perl -e 'while(<>){print +(split)[3], "\n"}' | gnuplot file.gp Where ...
1
vote
1answer
47 views

Find files and send to a temp file

Vim does support a nice option to accept a file with list of files that are then opened in a Quickfix list. Rough example, I find all files in ~ that are smaller than 1KB and pass this to Vim: find ~ ...
16
votes
5answers
957 views

How to trick a command into thinking its output is going to a terminal

Given a command that changes its behaviour when its output is going to a terminal (e.g. produce coloured output), how can that output be redirected in a pipeline while preserving the changed behaviour?...
4
votes
4answers
160 views

Use awk interactively through a pipe

Looking at this question, I've noticed that awk can't read user input if the file is being piped into standard input, but it does behave as expected if reading input from a file given as a command ...
-1
votes
1answer
99 views

are named pipes (mkfifo) the predecessor of RabbitMQ? [closed]

Is RabbitMQ, for inter process communication, like pipes and named pipes? How does RabbitMQ compare to named pipes? Except distributed systems. (RabbitMQ, for those who haven't encountered it, is an ...
3
votes
2answers
195 views

Process each line of output from `ping` immediately in pipeline

I have a few examples of different ways of extracting timing information from ping -c 10 google.com. For some of these pipelines, a line of output is produces every so often, just like the output of ...
10
votes
6answers
6k views

How do I pipe ls to grep, and delete the files filtered by grep?

I wanted to delete some package in my home file, but the filename was too long (google-chrome-stable_current_i386.deb). So, I decided to use the command ls|grep chrome|rm to pipe the files to grep to ...
2
votes
1answer
108 views

pipe the output of cat into a node script

I am an absolute beginner at UNIX scripting (and have searched here for something explaining how to do this in a simple way to no avail). I am trying to pipe the contents of a claimName.txt file find ...
0
votes
1answer
46 views

Trying to Use awk by implementating a condition and implication

I have a file containing 8 columns, One of my tasks is to get the values of Column 3 and if the value of the row is not equal to 123 the 8 coloumns will be printed and hence If column 3 is equal to ...
0
votes
1answer
19 views

Getting handle on item in a pipe and piping result

I have this pretty cool command that giving me a nice json representation of a nix package nix-env --json -qaA nixpkgs.wireshark | json_pp | less But what I really want in pseudo shell/OCaml nix-...
0
votes
2answers
56 views

Conditional grepping

I have a scenario to handle. Need help with that... I have a config file and its content looks something like below: Jobname|Type|Silo|Description #comment1 #comment2 job1|me|silo1|...
3
votes
1answer
69 views

Rsync contents of tar with remote server

I've got a number of backups as tar archives, and to do checking/testing of them I restore to some test machines using: cat archive.tar.gz | ssh server2 "cd /path/to/dir; tar zxvf -" Most of the ...
2
votes
1answer
41 views

who to get a count of “base subDir” without there subDirs?

All I am trying to do is get a count of direcrories on one leve of a tree. so Far I have this. find /media/data/temp1 -type d -exec basename {} \; which gives me this: temp1 Coldplay Greatest ...
2
votes
2answers
153 views

Avoid output buffering for a command group (curly braces) in bash script

I have a bash script containing a group of commands in curly braces { ... }. This group contains some initial echo commands and then one loop. At each iteration the loop executes various slow commands ...
2
votes
2answers
697 views

How to download an RPM package and install it in one line?

I was wondering whether using wget it was possible to download an RPM and then pipe it through sudo rpm -i to install it, in a single line? I realize I could just run: wget -c <URL> sudo rpm -i ...
2
votes
2answers
335 views

Curl url txt file, but grep each url seperately from single file

Ok, here's one for you guys that is puzzling me. So I have a text file with lots of url's in it. I'm using curl -K "$urls" > $output to spit the output to my output file. Now for the output of ...
1
vote
3answers
54 views

Pipes vs variable assignment

In what ways are pipes different from variable assignments? I know you can't use null bytes. What other examples are there? What about for piping in something like the following yes | cat? I notice ...
1
vote
2answers
153 views

pipes - Inserting pipe to middle of string [duplicate]

How would I make a pipe insert characters into the middle of a string? Here's my example (running Raspian): hostname -I | sudo python ~/testsite/manage.py runserver {the ip from the first command}:80 ...
1
vote
1answer
73 views

How are piped processes presented in ps output?

I have a following very simple script which watches /tmp directory for new files and output of inotifywait is piped to sed: #!/bin/bash /usr/local/bin/inotifywait -q -m /tmp --format %f | sed 's/a/b/...
5
votes
1answer
239 views

Sort output of awk except for first line?

This is a use case I am continually running into in parsing CSV files. When it is an inline awk script embedded in a shell script, I can use this workaround: myfile=$(mktemp) awk '(awk script here)' ...
14
votes
2answers
2k views

Why does using `yes` on bash pipelines *not* cause infinite loops?

According to its documentation, bash waits until all commands in a pipeline have finished running before continuing The shell waits for all commands in the pipeline to terminate before returning a ...
2
votes
2answers
153 views

pipe ls and awk to rsync

Trying to complete a script with the following steps: (1) select 1gig of most recently changed files across a number of subdirectories (2) preferably uses rsync to copy the files to local directory -- ...
0
votes
1answer
15 views

Creating a function that reads off an input list of files

I am new to UNIX/Linux and am in the process of writing bioinformatics pipelines. These pipelines take in input files and pass them through multiple packages. Say there is a list of files that goes ...
2
votes
1answer
67 views

How to trick an application expecting a file:// to read from stdin?

A file URI scheme makes it possible to specify an URI to a file stored locally, such as file:///home/demo/sample. Could it be used to specify stdin? For instance, with CURL: curl file:///home/demo/...
0
votes
0answers
20 views

Piped sequence - printing

I need to write a piped sequence of commands that prints out a list of all sub directories of a current directory. In order to get all sub directories of a current directory I use: ls -D Since I am ...
0
votes
1answer
46 views

Delete Empty Folders - recursive [duplicate]

I would like to clean up some folders. So I want to delete all folders which are empty or only contain other folders... First I tried find . -type d -delete but this lists the subfolders after ...
3
votes
3answers
575 views

How do I pipe the output of tar through mv?

A previous question of mine asked how to pipe downloaded files through tar, now I would like to know how to pipe the output of tar through mv. See I have this command at the moment: wget -c https://...
1
vote
1answer
239 views

Pass a value for an input prompt

I do unzip my.zip and if files already exists it asks replace myfile.ext [y]es, [n]o, [A]ll, [N]one, [r]ename: How can I force input for when the command prompts for input? I am aware unzip may have ...
3
votes
2answers
138 views

Extract compressed file by it's header as it is piped from stdout (locally or from remote location)?

I am sending compressed file with piping either local or from a network location. And on the receiving end, I would like to detect the type of compression and use the appropriate decompression utility ...
1
vote
0answers
56 views

Chaining grep commands [duplicate]

I'd like to find all xml files containing 'foo', but not containing 'bar'. The command I've put together looks like this: find -iname "*.xml" -print0 | xargs -0 grep -FlisZ "foo" | xargs -0 grep -...
1
vote
1answer
153 views

Extract characters from a serial port

I have a dummy question for you, I am trying to extract some lines from a file beginning by a special string of characters and to copy them into a file. I tried : cat /dev/ttyACM0 | grep "something" ...
1
vote
2answers
199 views

How do I pipe the files downloaded with wget through tar?

I would like to pipe tarballs (.tar.gz files in this case) I download with GNU Wget (wget) through tar -xzf (to decompress them, if this is unclear) but I don't know how. I have tried: wget -q -O- ${...
4
votes
1answer
72 views

Is this tail behavior in Grouping Commands specified by POSIX?

Using tail combined with other standard tools in Grouping Commands can make some powerful constructions. For example, to get the first and last line of a file: $ seq 10 > file $ { head -n1; tail -...
0
votes
1answer
111 views

Grep -v filter not working

I want to filter grep results by using grep -v option. But the output does not differ when using a particular pattern. contents of log.log: ERROR error EXCEPTION exception <STATUS>ERROR</...
2
votes
5answers
238 views

How do I cat a file I got from stdin without xargs

In an effort to learn the shell better, and without always having to resort to xargs, I have been trying to discover any other ways to do: find . -name *.tcl | xargs -I{} cat {} xargs make it feel ...
2
votes
2answers
52 views

Insert result of command into an executable (one command) [duplicate]

Is it possible to insert the result of a command (or even a chain of commands) into an executable in one command? Something like this: ./a.out < echo 0 Or is it necessary to do this: echo 0 >...
0
votes
1answer
67 views

Pretty tricky cmd [duplicate]

I was searching for a cmd to generate random passwords, and got stuck on this one : echo `< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c6` I've read the documentation of urandom, tr and head. But ...
2
votes
2answers
33 views

All lines not moving through pipe?

With slabtop I get the following output (50 lines): $slabtop -sc -o Active / Total Objects (% used) : 110864927 / 111473562 (99.5%) Active / Total Slabs (% used) : 2826375 / 2826375 (100.0%)...
6
votes
1answer
126 views

Function to simplify grep with an often used log

First off, sorry if this is a painfully newbish question! I'm trying make a function that simplifies grepping a log I have to work with on a regular basis. I'd like to use extended regexp with it ...
3
votes
2answers
60 views

can not access standard output from a command

I'm using this command format: cmd0 > >(cmd1) 2> >(cmd2) Cmd2 needs to echo data back to zenity as it processes some data, but zenity never hears the echos. It seems the echos are lost ...
1
vote
2answers
31 views

How can I open the file with the most matches for a given regex?

Let's say I have a directory ~/mydir that has a whole bunch of text files in it. I want to search for searchterm in this directory and then view the file that has the most matches. How can I do this ...