Chains the standard streams of a series of commands

learn more… | top users | synonyms

4
votes
2answers
386 views

How can I catch a command exit code for later, in a Makefile?

I have a makefile, with a test suite target, like so: rcheck: foo This shows the output from the foo command on stdout, and returns an exit code from foo which is handled by make as normal. ...
2
votes
1answer
155 views

Bash Named Pipe - Atomic Reading

I have set up one script to write to a named pipe, then four scripts to read the named pipe. For most of the entries, there are no problems, but when there is little processing to be done on the ...
1
vote
0answers
48 views

What to use instead of `buffer` for 1GB buffer?

buffer(1) seems to be old-ish and have hard-coded values preventing it to cache large amount of data. $ buffer -m 1G max_shmem 1 too low // it doesn't even understand gigabytes $ buffer -m 1000M ...
1
vote
1answer
46 views

pipe a file to a slow processing script question

Let's say I have a very large file and I want to process every line in that file by piping it to script. cat large_file.txt | python processor.py I'm not completely sure how the above operation ...
2
votes
1answer
51 views

pipe speed benchmark with machine readable output

I'm looking for a tool similar to pv which is producing machine readable output. And i cant believe that nobody did it bevore... I looked at the manpages of pv, pipemeter, dd and cpipe but all these ...
7
votes
4answers
166 views

Split an input for different command and combine the result

I know how to combine the result of different command paste -t',' <(commanda) <(commandb) I know pipe same input to different command cat myfile | tee >(commanda) >(commandb) Now ...
1
vote
1answer
139 views

Why does bash output data instead of executing, when a script is piped?

I'm running the following script on Ubuntu 14.04: #!/bin/bash apt-get purge -y nginx apt-get install -y nginx date When I run it like cat /tmp/script | bash, apt-get starts installing, then ...
20
votes
2answers
1k views

Pipes, how do data flow in a pipeline?

I don't understand how the data flows in the pipeline and hope someone could clarify what is going on there. I thought a pipeline of commands processes files (text, arrays of strings) in line by line ...
0
votes
2answers
139 views

ps output / pipeline execution order

I run ps aux | grep somethinghere. The output shows the grep somethinghere as a running process. My question is, shouldn't ps aux finish first, then grep somethinghere run on its output? Maybe there ...
4
votes
3answers
844 views

Piping a script with “read” to bash

I need to run a script by piping it through bash with wget(rather than running it directly with bash). $ wget -O - http://example.com/my-script.sh | bash It's not working because my script has read ...
1
vote
3answers
101 views

Tailing log files in two different hosts

I'm trying to tail two log files in different hosts and use the merged output as a single one. ssh HOST_1 "tail -f MY_LOG_FILE" I'm also using mkfifo to create a named pipe. mkfifo MY_PIPE The ...
3
votes
2answers
155 views

How to use tee to capture STDOUT from a code block to a filename defined within the block?

I want to send STDOUT for a script block to a file which name is defined by a variable within the block. However, when I use tee, it seems the variable outside the block doesn't exist anymore. Without ...
1
vote
1answer
65 views

Output Redirection

I am using the tee command to output the compilation errors of a program into a file along with the terminal. gcc hello.c | tee file.txt This is the command I have used. The compilation errors are ...
0
votes
1answer
53 views

exit status and no output

I have a command such that bar > /dev/null and I want to know the exit status of bar. I read some posts su about ${PIPESTATUS[0]} but this works when one pipes the output via | and I can't make ...
2
votes
2answers
132 views

What exactly is the difference between pipes and streams?

I read about pipes and streams and I’m still confused on how it’s implemented. A program is started and it reads data from “standard input” stream (stdin), which is where the keyboard sends data to. ...
0
votes
0answers
30 views

Pipe to `ls` Not Working as Expected [duplicate]

When I issue: [dave@dhost ~]$ which tar | ls -l I get the output of ls -l of the pwd. I expected to get: -rwxr-xr-x. 1 root root 395472 Feb 21 2013 /bin/tar Why doesn't the pipe work as I ...
4
votes
3answers
303 views

Using | pipe character from a $variable makes it treat as just another argument in bash; how to escape it?

I have a bash script like this export pipedargument="| sort -n" ls $pipedargument But it gives the error ls: |: No such file or directory ls: sort: No such file or directory It seems to be ...
2
votes
2answers
221 views

Feed source command with a pipe

Previously I used source command like this: source file_name But what I'm trying to do is this: echo something | source Which doesn't work.
2
votes
3answers
74 views

Pipe files downloaded with wget into a function

I'm downloading a bunch of images with wget and I want to rm the files if they are empty with a function to which I can pass the file. My question is how do I pipe the images to a function like this: ...
1
vote
1answer
119 views

preserving colors when piping yaourt through sort

Can sort preserve piped in colors? I'm trying to modify yaourt -Qu | sort so I get color output. Tried googling this but I couldn't find a general way nor a special way to do this for sort.
0
votes
1answer
40 views

Why doesn't this tee with process substitution produce the 1st and chosen lines? [duplicate]

What I want is printing the output the first line ( table head ) of ps aux and the grep result. After search, I come up with following. ps aux | tee >(head -1 > /dev/tty) | grep mongo But I ...
1
vote
1answer
28 views

How do I make an alias to substitute single word in a piped command? [duplicate]

I use aliases a lot but right now only for use cases like alias i='sudo apt-get install -y'. I often would like to add an alias in the following form: alias cmd='echo [something] >> ...
3
votes
3answers
153 views

Redirecting stdout to terminal and file without using a pipe?

I have code, that goes something like this: #!/bin/bash VAR=0 func() { VAR=$((VAR+1)) echo 'Logging information.' } func 2>&1 | tee 'log.txt' echo "Should be 1: ${VAR}" When ...
3
votes
2answers
191 views

Am I hitting a race condition in bash?

I have a script which compares the output of a command with the output of the same command as it was ran previously, it works most of the time, but every now and then it doesn't work as expected. ...
2
votes
3answers
613 views

source /dev/stdin doesn't work as expected

Let's start with simple tests that work for me to check that source /dev/stdin can be used at all. # echo -ne 'echo a\necho b\n' | source /dev/stdin a b Now I would like to source an actual ...
4
votes
1answer
139 views

cmd2 `cmd1` vs cmd1 | xargs cmd2

I'm trying to find a file with find and then pass the found path to sqlite3. I'm only dropped into the sqlite> prompt if my command is of the form: sqlite3 `find . -type f -iname "*.db" | head ...
0
votes
1answer
69 views

Force ping to write before interrupt

I am trying to get ping to continuously write some times to a file. When I run $ ping 10 10.1.10.28 | perl -ne '/time=(\d+\.\d+)/ && print "$1\n"' It returns one number every several ...
6
votes
4answers
596 views

Back to back Pipes into a Command

Is there a way to pipe back to back values into a command? e.g. If I can command.pl and want to pipe in response, then y to confirm (because it doesn't have a way to auto confirm)? I've tried echo ...
0
votes
1answer
199 views

Using tee or pipe to log console output will cause my program not to start [duplicate]

I'm running hostapd on my raspberry PI to let it function as an accesspoint. It's working fine with the following command: sudo hostapd /etc/hostapd/hostapd.conf This will start hostapd and log all ...
2
votes
1answer
127 views

'Tar' the result of a 'find', preserving the directory structure

I want to tar all the *.txt files that I get as the result of a find command, that exist in a directory having a tree structure like this: Directory_name dir1 file1.pdf file1.txt dir2 ...
3
votes
1answer
130 views

Problem with getting interactive input in while read loop

I was doing something like the following minimal example: #! /bin/sh # $1 = file containing list of files to sync # $2 = source dir ; $3 = target dir cat "$1" | while read f do cp -i "$2"/"$f" ...
2
votes
2answers
247 views

How to read from a named pipe after the writer process exits?

I want to use named pipes as asynchrnonous task queues (JMS-like, perhaps). For example, suppose a client sends some data that I wish to insert into a database, but I want the client to disconnect ...
2
votes
1answer
93 views

Problems with find, xargs and egrep

I this is what I what I'm trying to end up with (except working) find ./ -mindepth 1 -type f -mtime +60 -print0 | xargs -0 egrep -vZ 'vvv|iii' What am I doing wrong? $ ll total 0 -rw-rw-r-- 1 ...
3
votes
2answers
490 views

Less resource hungry alternative for piping `cat` into gzip for huge files

I have some files of which some are very large (like several GB), which I need to concatenate to one big file and then zip it, so something like this: cat file1 file2 file3 file4 | gzip > ...
1
vote
2answers
1k views

awk - Group by and sum column values

I have command to list system process by memory usage: ps -A --sort -rss -o comm,pmem Which list a table like COMMAND %MEM firefox 28.2 chrome 5.4 compiz 4.8 ...
4
votes
2answers
338 views

How to read user input when using script in pipe

General problem I want to write a script that interacts with the user even though it is in the middle of a chain of pipes. Concrete example Concretely, it takes a file or stdin, displays lines ...
3
votes
2answers
758 views

Bash: cannot break out of piped “while read” loop; process substitution works

I intend to pipe the output of a program into a while read VAR loop and break when a pattern is found, but it doesn't. Proof of concept: inotifywait -qm -e create . | while read line; do echo $line; ...
1
vote
2answers
83 views

Capturing data from a Fluke 1620a via netcat

I am trying to pull temperature and humidity data from Fluke DewK 1620a thermo-hygrometers and write the temp/humidity readings to a log. I can connect to the device via netcat with a simple 'nc ...
2
votes
2answers
67 views

How to pipe a the output of find to another find invocation

Here's what I'm trying to do find . -maxdepth 1 -type d -print0 | xargs -0 -P4 find -name 'war' -type d I would like to create a list of directory names, null-terminated, and then pass this list to ...
1
vote
2answers
202 views

How does Bash pipe large amounts of data?

Let's say you want to cat the contents of a really big file, but want to view it a few bits at a time. Let's say one were to do the following: $ cat /dev/sda1 | less As a programmer of languages ...
1
vote
3answers
42 views

Command that passes its input through to output AND performs command?

I'm trying to use the quote program to make a little shell script that'll keep updating the stock prices every second. quote has to access the Internet and therefore has a small delay when it runs. I ...
1
vote
2answers
1k views

Combination of echo, pipe and cat

I have a file name called temp.csv in my script one of the intermediate step is echo "some info" > final.csv | cat temp.csv >> final.csv at times the file final.csv is created without ...
1
vote
1answer
79 views

Using find -exec and having trouble with redirecting standard error

I have an individual.sh script that pipes standard error to a grep command, like this: FOUND=$(tesseract "$i" out -psm 0 |& grep -ci 'Orientation in degrees: [^0]') If I run the script like ...
2
votes
1answer
35 views

Confused about piping commands from find to commandX?

I have a script, run.sh, that looks like this: #!/bin/bash FILES=$(find corpus/ -type f) for i in $FILES do ./individual.sh $i done It runs without problem. I want to do away with the run ...
1
vote
1answer
144 views

Pipes & redirection binding precedence with disjuncts, conjuncts etc?

I know of the relative binding precedence of operators ‘;’, ‘&’, ‘&&’, or ‘||’ http://www.gnu.org/software/bash/manual/bashref.html#Lists but when pipes come into the picture along with ...
3
votes
1answer
92 views

How to find the file corresponding to a pipe opened by a process? [duplicate]

I created this script, which listens for characters received on a hardcoded pipe, and tries to eval those characters. I call my script loop_executor.sh This is all it contains: !/bin/bash while : do ...
1
vote
1answer
119 views

Redirect output to stdout and pipe to a binary

I have a nice PS1 line in my .bash_profile, and I want to copy it to another machine. So I want to view it AND copy it to my clipboard. I can't figure out how to string the commands to do this ...
2
votes
1answer
265 views

Advantages of using named pipes and sockets rather than temporary files

I have two cooperating programs. One program just writes its output to a file and the other one then reads from the file and spits the data out for the front end to work with. I have been reading ...
3
votes
2answers
521 views

How can a bash script write its standard output from within a pipeline? [duplicate]

I have a bash script that calls a function. The function, amongst other things, executes a pipeline that sinks its output. To simplify it, here is a contrived example: #!/bin/bash func() { ls "$@" ...
1
vote
2answers
253 views

Append a file, output to screen and redirect from stderr all at once, without tee

I'm running a time dd command in a shell script, and I'd like to output the results to a file, and print it out on the screen. The line that I'm currently running is: (time dd of=$dest_filepath ...