A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

learn more… | top users | synonyms

1
vote
1answer
39 views

Array from piped commands fails

First off, apologies if this has been asked before, I searched both here and StackOverflow, tried the man pages, and I'm still drawing a blank. I'm trying to write a script that will auto-mount our ...
1
vote
2answers
77 views

How to redirect dd to pv? [duplicate]

This is my dd command which I need to modify: dd if=/tmp/nfs/image.dd of=/dev/sda bs=16k Now I would like to use pv to limit the speed of copying from NFS server. How can I achieve that? I know ...
1
vote
1answer
32 views

conditional coloring using perl expression

I have two commands in perl and one test file test.txt: $ cat test.txt aaa bla bbb foo #ccc bar ddd 444 First perl expression colorizes any occurrence of aaa or bbb with red color: cat test.txt | ...
2
votes
2answers
42 views

Pipe Find Results for '*.squashfs' to unsquashfs

I am looking for something that will find the squashed filesystem, and the pass the output to unsquash the fs and it would have to be an absolute path outside the squashfs. example ...
14
votes
5answers
1k views

Is this a bug in bash? `return` doesn't quit function if called from a pipe

I'm been having some weird problems with bash lately. While trying to simplify my script, I came up with this small piece of code: $ o(){ echo | while read -r; do return 0; done; echo $?;}; o 0 $ ...
6
votes
2answers
463 views

Send the password through stdin in `ssh-copy-id`

I need to automate some identity deployments, ideally using ssh-copy-id. I'm trying to provide the password through stdin, which is possible on ssh by using the -S flag. I'm aware that I can send ...
1
vote
0answers
54 views

UDEV-rule:Open new gnome-terminal window on usb-plugin AND activate it for stdin/stdout/stderr?

My attempt is this: If a certain USB-device is plugged in, I want to open a gnome-terminal window in my desktop-environment. This newly opened window must be able to be interactive, this means, the ...
5
votes
1answer
259 views

logging to /dev/xconsole does not work properly

I am using rsyslog, and I want to send logs to /dev/xconsole. I have following relevant line in /etc/rsyslog.conf *.*;cron.!=info;mail.!=info |/dev/xconsole This works only partially. It looks ...
2
votes
2answers
53 views

why my first pipe operation didn't remove newline character?

My nextermCensus.lis contains newline character and I am only interested in the first line, I'd like to remove newline character. but it doesn't work. so I have to use second method to remove it. ...
0
votes
1answer
73 views

Find mails, pipe them to php and delete after

I have a script that saves to DB content of piped to it file. cat /mailpath/Maildir/cur/mailfile | php -q /scriptpath/mailPipe.php I need a shell script or oneliner to find all files in Maildir, ...
2
votes
3answers
103 views

Why does using pipe '|' terminate the second process, is there a way to prevent it?

Just to give some context, I'm trying to run this command echo "set hive.execution.engine=tez;" | hive hive terminates as soon as "set hive.execution.engine..." has been entered into the hive ...
0
votes
1answer
57 views

Fetch second and fourth column from a file

Problem Statement I am not able to get the values in second and fourth column. Tried grep -i "AUGC\|AACN\|AACY" output.csv | cut -d ',' 2,4 DATA ...
2
votes
3answers
38 views

Is there a way to colourize the output of a command without using bashrc/bash_profile?

I have made a program where I can read several logs from different programs, I want to colourize certain terms in the output , so I can find the more interesting parts. (I will not use grep or ...
4
votes
2answers
163 views

SSH: Provide additional “pipe” fds in addition to stdin, stdout, stderr

When connecting to a host with SSH, usually three "pipes" are provided between host and guest, for stdin, stdout, and stderr. Is there a command-line option to create forwards for additional file ...
0
votes
1answer
31 views

Having extra parameters in piping or redirection

I have two programs and I want to pipe them into each other but they take extra parameters: prog1 <inputfile> <outputfile> min max prog2 <inputfile> <outputfile> min max ...
0
votes
1answer
115 views

cat SQL files and pass to import schema

I am trying to import a series of .SQL files which map one file per DB object into mysql without calling mysql 50 times, something like: mysql -u root < cat schema/db1/*.sql This isn't working ...
26
votes
5answers
2k views

Why pipe to cat only to redirect? [duplicate]

I occasionally see things like: cat file | wc | cat > file2 Why do this? When will the results (or performance) differ (favourably) from simply: cat file | wc > file2
8
votes
4answers
362 views

How can I filter the contents of a tar file, producing another tar file in the pipe?

Consider a single tar file from an external system which contains some directories with various attributes which I want to retain such as permissions, mtimes, etc. How can I easily take a subset of ...
3
votes
1answer
181 views

How do I append an item to an array in a pipeline? [duplicate]

This script should simply add a value to an array through a loop and then show all items of an array. #!/bin/bash data_file="$1" down=() counter=0 cat $data_file | while read line; do \ ...
6
votes
1answer
114 views

Extra output on input redirection

I have a folder with three files: $ ls a b c If I pipe the output of ls to wc, I get the right result: $ ls | wc -l 3 However, when I specify the input to wc as the output of ls, I get extra ...
0
votes
1answer
850 views

piping a password located in a file to the ssh command [duplicate]

So I need to login as a lot of different users to a server using ssh and I'm tired of copying the password from the files, which contain the passwords for the users, each time. I can't change it to ...
0
votes
1answer
205 views

How to use pipe to assign a variable [duplicate]

I am new to bash script. Probably this is very easy but for me it's not working. How can I assign value of $x1+$x2 to bc by piping. here in my code $bc holds no value. echo -n 'value x1 : ' read x1 ...
5
votes
3answers
783 views

Behaviour of “|” pipe in linux with “>” output redirection

I am executing command ls > a.txt | sort > b.txt This command is doing the below things : executing ls sorting it creating a.txt and storing sorted output to a.txt creating b.txt , but its ...
-2
votes
2answers
570 views

Pipe Named Fifo [closed]

I am implementing a named pipe to write and read data between two processes in cpp. The first process acquires a feature point from an image in real time while the second reads this feature point. It ...
1
vote
1answer
100 views

unzip a file and pipe to “ddrescue” command

I have a zipped image file, and need use "ddrescue" command instead "dd" to do something like this: unzip -p ubuntuXXXX.img.zip | sudo dd bs=2M of=/dev/sdb But "ddrescue" needs the image file as ...
3
votes
4answers
74 views

Who forks the processes of piped commands in the shell?

Consider this command in the shell : ls|less Now we know that ls and less are run as two different processes. We know that the shell forks and creates a child process which load the ls program using ...
0
votes
4answers
116 views

piping commands not working as expected I need explanation

I am trying to do the below touch ~/scratches/scr6.txt | cat ~/scr.txt > ~/scratches/scr6.txt | cat /dev/null > ~/scr.txt I don't know why scr6.txt is always empty, however doing the below ...
3
votes
1answer
463 views

Round-tripping text through zip, unzip fails with 'extra bytes' error on OS X [duplicate]

I'm trying to zip streaming text, and unzip it later. A minimal example is: echo "Hey, could you zip this for me?" | zip hello.zip - ; unzip -p hello.zip This fails, with the following output: ...
1
vote
1answer
176 views

Deadlock on read/wait [closed]

My process deadlocks. master looks like this: p=Popen(cmd, stdin=PIPE, stdout=PIPE) for ....: # a few million p.stdin.write(...) p.stdin.close() out = p.stdout.read() p.stdout.close() exitcode = ...
14
votes
2answers
495 views

Resource usage using pipe and here string

We can get the same result using the following two in bash, echo 'foo' | cat and cat <<< 'foo' My question is what are the difference between these two as far as the resources used are ...
1
vote
1answer
48 views

CD with redirect to logger does not work

On Linux, the following cd works fine so cds into tmp: / # cd /tmp tmp # But this does not work (stays at root, nothing appears in /var/log/messages). Why not? / # cd /tmp 2>&1 | ...
2
votes
0answers
216 views

Pipe a .BIN file into XModem transfer

I need to send a .bin file to a modem via XModem transfer for firmware upgrading. If the modem is connected to a local computer, the command would be the following: sz -X -k -b -vvv - ...
0
votes
1answer
138 views

Bash : rewriting a file which is read in same line

I'm working on a bash script for partially automating router connection in command line. I'm a newbie in Bash. This is part where I have a problem: #!/bin/bash STRING2="ESSID" MYPWD=$(pwd) sudo ...
7
votes
2answers
394 views

How to send all output to `logger` in POSIX shell?

I'd like to log standard output and standard error separately in .xprofile using logger. In Bash I think that would look something like this: exec 1> >(logger --priority user.notice --tag ...
1
vote
2answers
134 views

Piping contents of multiple files between programs while keeping contents separate

I would like to read in a number of files and pipe their output to subsequent programs while still maintaining them as individual pipelines of data. program1 *.txt | program2 | program3 folder I ...
0
votes
1answer
75 views

ssh over unix pipe or pty device [closed]

Ok, I don't even know how to put this into words. I have a /dev/myDevice which automatically connects to some ssh server when opened. How do I connect to it with ssh? Is this even sane? If it's insane ...
4
votes
1answer
81 views

Joined pipelines

Considering a routine such as this one: alpha() { echo a b c |tr ' ' '\n'; } which outputs a stream, I would like to take the output stream, transform it, and paste it with the original output ...
3
votes
1answer
573 views

Possible to pipe command using su via ssh

I'm trying to write a python script to remotely update the login passwords for a set of account I administer - I can already successfully use python to pipe remote commands via the stdin of an ssh ...
0
votes
4answers
292 views

How to delete files from a directory based on the contents of a gzipped tar file?

Given that I: have a directory that contains over 1000 files have a gzip'ed tar file that contains a subset of those files (x.tgz) What is the single command line (if it is possible) that will ...
1
vote
2answers
390 views

How to while read line with a pipe

I am currently trying read a csv file with BASH. The 10th [$f10] value of the csv file must be greater then 0 for a given machine class and machine ID [m_class] and [m_id] (these are user defined) . ...
0
votes
1answer
98 views

netstat pipe count host

Unix pipe problem: how to process multiline input to output the line and insert some piped processes. echo "111 22222222" | awk '{print $1 " " $2 " " piped_processes_using_2nd_argument}' A bit of ...
5
votes
3answers
414 views

How to forward between processes with named pipes?

The /tmp/in, /tmp/out and /tmp/err are named pipes, already created and opened by some process (for reading, writing and writing, respectively). I would like to create a new process that pipes its ...
1
vote
4answers
550 views

Grep for a string in file without using pipe

I want to grep for a word in a file in the last n lines without using the pipe. grep <string> filename enables to search the filename for a string. But, I want to search for a string in the ...
2
votes
4answers
363 views

How to influence a bash while loop with a timed global variable?

I have a while loop in a bash script which should do something different at the beginning and at every 5 second interval. Any previous loop is allowed to complete. The 5s interval is indicated by the ...
1
vote
4answers
166 views

optimize command with or and pipe to parse the output of ifconfig

I have this command line, but it's doubling the grep and awk ifconfig eth1 2> /dev/null | grep "inet " | awk '{gsub("addr:","",$2); print $2 }' || ifconfig eth0 2> /dev/null | grep "inet " | ...
5
votes
1answer
804 views

Logging bash output to a file

I'm trying to build vlc which is rather complicated and dependent on having the right packages installed, and keeps choking on errors. To trace all my steps I wish to output what I'm doing to a ...
0
votes
2answers
35 views

Saving a piped value went wrong

I installed jq to handle some JSON with bash. But somehow piping the JSON string to jq in combination with saving it's output to another variable went wrong. TEST='{"foo": "bar"}' PB_SIG=$TEST | jq ...
1
vote
2answers
256 views

viewing printf after pipe (or subshell?)

Ok, this is hard to distill to a single question because it really depends on how I'm going to troubleshoot it. The following script is meant to pipe mount.cifs commands to a while loop that enters ...
2
votes
2answers
209 views

Difference of using () and $() to execute a series of commands

I am currently trying to make a script that creates bytes that will be piped as input to netcat. Here is the idea of the script: (perl -e "print \"$BYTES\x00\"; cat file; perl -e "print \"More ...
1
vote
3answers
123 views

How to write a script that can take input from stdout

I want to be able to write a script that can take stdout as an argument, if anything is piped into it (ultimately, I would like it to be polymorphic) - The trouble is, I have searched and searched ...