Chains the standard streams of a series of commands

learn more… | top users | synonyms

8
votes
4answers
608 views

Spaces next to pipes

I've seen history | grep blah and history |grep blah; and history|grep blah also works, though no one ever seems to use it. Is there any significance in the spaces (e.g. piping to/from different ...
0
votes
1answer
24 views

explain this shell command with redirection and pipe

I can understand the statements like. bzip2 --version >/tmp/list_right 2>/tmp/list_error bzip2 --version >/tmp/list 2>&1 But I don't understand the statement: bzip2 --version ...
0
votes
0answers
25 views

Combine the output of while loop into tar

I'm trying to create a script that mysqldumps all my databases, creating an individual .sql file for each database and combines the output into backup-DATE.tar.gz. I've gotten as far as being able to ...
0
votes
1answer
18 views

Saving from arecord on a raspberry pi across a network to a mac

I've currently got a usb mic plugged into a Raspberry Pi B+ and am using the following code to record data: arecord --buffer-time=5000000 -D plughw:1,0 -f cd -t raw | lame -r - stream.mp3 I then ...
3
votes
1answer
237 views

Chaining pipe | with &&

I have this command where I want to filter make output: cd /app && make && sudo nginx -g 'daemon off;' What is the correct way to insert make | pv -q -L 100 here?
3
votes
1answer
140 views

eval limitation with piped commands

We have a shell script that builds a long piped command chain in a variable and executes it with eval (the following code is simplified to the essential): cmd="cat /some/files | grep -v \"this\" | ...
12
votes
3answers
432 views

No colored output in less for the ls command

If I run ls I get colored output which I find pretty handy for quickly getting a glance of the kind of file. When I try to pipe it to less even with the -r and -R flags the coloring always get lost. I ...
0
votes
3answers
46 views

Reuse output from command 1 in command 2 using a pipe

I've got a command in linux which returns a list of numbers. Now I need this numbers to form a directory path and cat the file located at that path. For example: myCommand returns: 1 10 30 40 And ...
1
vote
1answer
53 views

How to pipe scp/ssh file transfer(pull) through tar (remote->local transfer initiated from local machine)

I would like to pipe an scp pull operation(sample without tar below) through tar, so that only one tar file is written for about 10 directories: user@localhost:/home/user> scp -r ...
1
vote
1answer
27 views

Non-block poll from two pipes?

Here is what I want to do.. mkfifo infrequentlog frequentlog sort -m infrequentlog frequentlog > mylog funcA > infrequentlog funcB > frequentlog So that when monitor tail, I can see the ...
3
votes
2answers
36 views

How can I catch a command exit code for later, in a Makefile?

I have a makefile, with a test suite target, like so: rcheck: foo This shows the output from the foo command on stdout, and returns an exit code from foo which is handled by make as normal. ...
2
votes
1answer
53 views

Bash Named Pipe - Atomic Reading

I have set up one script to write to a named pipe, then four scripts to read the named pipe. For most of the entries, there are no problems, but when there is little processing to be done on the ...
0
votes
0answers
23 views

What to use instead of `buffer` for 1GB buffer?

buffer(1) seems to be old-ish and have hard-coded values preventing it to cache large amount of data. $ buffer -m 1G max_shmem 1 too low // it doesn't even understand gigabytes $ buffer -m 1000M ...
1
vote
1answer
26 views

pipe a file to a slow processing script question

Let's say I have a very large file and I want to process every line in that file by piping it to script. cat large_file.txt | python processor.py I'm not completely sure how the above operation ...
2
votes
1answer
19 views

pipe speed benchmark with machine readable output

I'm looking for a tool similar to pv which is producing machine readable output. And i cant believe that nobody did it bevore... I looked at the manpages of pv, pipemeter, dd and cpipe but all these ...
5
votes
3answers
99 views

Split an input for different command and combine the result

I know how to combine the result of different command paste -t',' <(commanda) <(commandb) I know pipe same input to different command cat myfile | tee >(commanda) >(commandb) Now ...
1
vote
1answer
95 views

Why does bash output data instead of executing, when a script is piped?

I'm running the following script on Ubuntu 14.04: #!/bin/bash apt-get purge -y nginx apt-get install -y nginx date When I run it like cat /tmp/script | bash, apt-get starts installing, then ...
20
votes
2answers
1k views

Pipes, how do data flow in a pipeline?

I don't understand how the data flows in the pipeline and hope someone could clarify what is going on there. I thought a pipeline of commands processes files (text, arrays of strings) in line by line ...
0
votes
2answers
52 views

ps output / pipeline execution order

I run ps aux | grep somethinghere. The output shows the grep somethinghere as a running process. My question is, shouldn't ps aux finish first, then grep somethinghere run on its output? Maybe there ...
3
votes
3answers
184 views

Piping a script with “read” to bash

I need to run a script by piping it through bash with wget(rather than running it directly with bash). $ wget -O - http://example.com/my-script.sh | bash It's not working because my script has read ...
1
vote
3answers
54 views

Tailing log files in two different hosts

I'm trying to tail two log files in different hosts and use the merged output as a single one. ssh HOST_1 "tail -f MY_LOG_FILE" I'm also using mkfifo to create a named pipe. mkfifo MY_PIPE The ...
3
votes
2answers
88 views

How to use tee to capture STDOUT from a code block to a filename defined within the block?

I want to send STDOUT for a script block to a file which name is defined by a variable within the block. However, when I use tee, it seems the variable outside the block doesn't exist anymore. Without ...
1
vote
1answer
49 views

Output Redirection

I am using the tee command to output the compilation errors of a program into a file along with the terminal. gcc hello.c | tee file.txt This is the command I have used. The compilation errors are ...
0
votes
1answer
26 views

exit status and no output

I have a command such that bar > /dev/null and I want to know the exit status of bar. I read some posts su about ${PIPESTATUS[0]} but this works when one pipes the output via | and I can't make ...
2
votes
2answers
58 views

What exactly is the difference between pipes and streams?

I read about pipes and streams and I’m still confused on how it’s implemented. A program is started and it reads data from “standard input” stream (stdin), which is where the keyboard sends data to. ...
0
votes
0answers
30 views

Pipe to `ls` Not Working as Expected [duplicate]

When I issue: [dave@dhost ~]$ which tar | ls -l I get the output of ls -l of the pwd. I expected to get: -rwxr-xr-x. 1 root root 395472 Feb 21 2013 /bin/tar Why doesn't the pipe work as I ...
4
votes
3answers
161 views

Using | pipe character from a $variable makes it treat as just another argument in bash; how to escape it?

I have a bash script like this export pipedargument="| sort -n" ls $pipedargument But it gives the error ls: |: No such file or directory ls: sort: No such file or directory It seems to be ...
2
votes
2answers
95 views

Feed source command with a pipe

Previously I used source command like this: source file_name But what I'm trying to do is this: echo something | source Which doesn't work.
2
votes
3answers
53 views

Pipe files downloaded with wget into a function

I'm downloading a bunch of images with wget and I want to rm the files if they are empty with a function to which I can pass the file. My question is how do I pipe the images to a function like this: ...
1
vote
1answer
48 views

preserving colors when piping yaourt through sort

Can sort preserve piped in colors? I'm trying to modify yaourt -Qu | sort so I get color output. Tried googling this but I couldn't find a general way nor a special way to do this for sort.
0
votes
1answer
28 views

Why doesn't this tee with process substitution produce the 1st and chosen lines? [duplicate]

What I want is printing the output the first line ( table head ) of ps aux and the grep result. After search, I come up with following. ps aux | tee >(head -1 > /dev/tty) | grep mongo But I ...
1
vote
1answer
18 views

How do I make an alias to substitute single word in a piped command? [duplicate]

I use aliases a lot but right now only for use cases like alias i='sudo apt-get install -y'. I often would like to add an alias in the following form: alias cmd='echo [something] >> ...
3
votes
3answers
93 views

Redirecting stdout to terminal and file without using a pipe?

I have code, that goes something like this: #!/bin/bash VAR=0 func() { VAR=$((VAR+1)) echo 'Logging information.' } func 2>&1 | tee 'log.txt' echo "Should be 1: ${VAR}" When ...
3
votes
2answers
93 views

Am I hitting a race condition in bash?

I have a script which compares the output of a command with the output of the same command as it was ran previously, it works most of the time, but every now and then it doesn't work as expected. ...
2
votes
3answers
150 views

source /dev/stdin doesn't work as expected

Let's start with simple tests that work for me to check that source /dev/stdin can be used at all. # echo -ne 'echo a\necho b\n' | source /dev/stdin a b Now I would like to source an actual ...
4
votes
1answer
112 views

cmd2 `cmd1` vs cmd1 | xargs cmd2

I'm trying to find a file with find and then pass the found path to sqlite3. I'm only dropped into the sqlite> prompt if my command is of the form: sqlite3 `find . -type f -iname "*.db" | head ...
0
votes
1answer
30 views

Force ping to write before interrupt

I am trying to get ping to continuously write some times to a file. When I run $ ping 10 10.1.10.28 | perl -ne '/time=(\d+\.\d+)/ && print "$1\n"' It returns one number every several ...
6
votes
4answers
474 views

Back to back Pipes into a Command

Is there a way to pipe back to back values into a command? e.g. If I can command.pl and want to pipe in response, then y to confirm (because it doesn't have a way to auto confirm)? I've tried echo ...
0
votes
1answer
73 views

Using tee or pipe to log console output will cause my program not to start [duplicate]

I'm running hostapd on my raspberry PI to let it function as an accesspoint. It's working fine with the following command: sudo hostapd /etc/hostapd/hostapd.conf This will start hostapd and log all ...
2
votes
1answer
51 views

'Tar' the result of a 'find', preserving the directory structure

I want to tar all the *.txt file, that I get from the result of a find, that exist into a directory having a tree like this: Directory_name dir1 file1.pdf file1.txt dir2 file2.pdf ...
3
votes
1answer
69 views

Problem with getting interactive input in while read loop

I was doing something like the following minimal example: #! /bin/sh # $1 = file containing list of files to sync # $2 = source dir ; $3 = target dir cat "$1" | while read f do cp -i "$2"/"$f" ...
2
votes
2answers
142 views

How to read from a named pipe after the writer process exits?

I want to use named pipes as asynchrnonous task queues (JMS-like, perhaps). For example, suppose a client sends some data that I wish to insert into a database, but I want the client to disconnect ...
2
votes
1answer
47 views

Problems with find, xargs and egrep

I this is what I what I'm trying to end up with (except working) find ./ -mindepth 1 -type f -mtime +60 -print0 | xargs -0 egrep -vZ 'vvv|iii' What am I doing wrong? $ ll total 0 -rw-rw-r-- 1 ...
3
votes
2answers
114 views

Less resource hungry alternative for piping `cat` into gzip for huge files

I have some files of which some are very large (like several GB), which I need to concatenate to one big file and then zip it, so something like this: cat file1 file2 file3 file4 | gzip > ...
1
vote
2answers
338 views

awk - Group by and sum column values

I have command to list system process by memory usage: ps -A --sort -rss -o comm,pmem Which list a table like COMMAND %MEM firefox 28.2 chrome 5.4 compiz 4.8 ...
3
votes
2answers
137 views

How to read user input when using script in pipe

General problem I want to write a script that interacts with the user even though it is in the middle of a chain of pipes. Concrete example Concretely, it takes a file or stdin, displays lines ...
3
votes
2answers
247 views

Bash: cannot break out of piped “while read” loop; process substitution works

I intend to pipe the output of a program into a while read VAR loop and break when a pattern is found, but it doesn't. Proof of concept: inotifywait -qm -e create . | while read line; do echo $line; ...
1
vote
2answers
44 views

Capturing data from a Fluke 1620a via netcat

I am trying to pull temperature and humidity data from Fluke DewK 1620a thermo-hygrometers and write the temp/humidity readings to a log. I can connect to the device via netcat with a simple 'nc ...
2
votes
2answers
55 views

How to pipe a the output of find to another find invocation

Here's what I'm trying to do find . -maxdepth 1 -type d -print0 | xargs -0 -P4 find -name 'war' -type d I would like to create a list of directory names, null-terminated, and then pass this list to ...
1
vote
2answers
113 views

How does Bash pipe large amounts of data?

Let's say you want to cat the contents of a really big file, but want to view it a few bits at a time. Let's say one were to do the following: $ cat /dev/sda1 | less As a programmer of languages ...