Chains the standard streams of a series of commands

learn more… | top users | synonyms

21
votes
4answers
946 views

Is it possible to redirect the output of a command into more than one command?

As far as I know, I can use the tee command to split the standard output onto the screen and further files: command -option1 -option2 argument | tee file1 file2 file3 Is it possible to redirect ...
0
votes
2answers
93 views

Will “tail -100 | sed -n 1p” only read one line?

I'm searching backwards from the end of a file, it's probably easiest to explain by showing my code: enough=0 numberOfLines=100 fileName="/path/to.file" # Tail the last $numberOfLines lines; if the ...
0
votes
3answers
98 views

grep randomly appearing and disappearing in ps aux (ps aux | grep python) [duplicate]

This isn't an actual problem - but more of a curious question, when I run while true; do ps aux | grep abc; echo done; done I get the following: user 29733 0.0 0.0 11748 924 pts/1 R+ ...
3
votes
1answer
90 views

Pipe command output to Yad and also log the output to a logfile

Need to have the process of command typically an update to be shown using yad and at the same time log all the output to a given log file set up. This is what I have apt-get update | yad ...
3
votes
2answers
296 views

Named pipes: several experiments leads to confusion [closed]

I've come across various articles and SO questions and I am still confused about something that I use on daily basis, but never realized how confusing it can be. I am experimenting with (named) pipes ...
2
votes
4answers
88 views

Completely buffer command output before piping to another command?

Is there a way to only execute a command after another is done without a temp file? I have one longer running command and another command that formats the output and sends it to a HTTP server using ...
10
votes
4answers
871 views

Spaces next to pipes

I've seen history | grep blah and history |grep blah; and history|grep blah also works, though no one ever seems to use it. Is there any significance in the spaces (e.g. piping to/from different ...
0
votes
1answer
61 views

explain this shell command with redirection and pipe

I can understand the statements like. bzip2 --version >/tmp/list_right 2>/tmp/list_error bzip2 --version >/tmp/list 2>&1 But I don't understand the statement: bzip2 --version ...
0
votes
1answer
105 views

Combine the output of while loop into tar

I'm trying to create a script that mysqldumps all my databases, creating an individual .sql file for each database and combines the output into backup-DATE.tar.gz. I've gotten as far as being able to ...
0
votes
1answer
113 views

Saving from arecord on a raspberry pi across a network to a mac

I've currently got a usb mic plugged into a Raspberry Pi B+ and am using the following code to record data: arecord --buffer-time=5000000 -D plughw:1,0 -f cd -t raw | lame -r - stream.mp3 I then ...
3
votes
1answer
342 views

Chaining pipe | with &&

I have this command where I want to filter make output: cd /app && make && sudo nginx -g 'daemon off;' What is the correct way to insert make | pv -q -L 100 here?
4
votes
1answer
198 views

eval limitation with piped commands

We have a shell script that builds a long piped command chain in a variable and executes it with eval (the following code is simplified to the essential): cmd="cat /some/files | grep -v \"this\" | ...
12
votes
3answers
757 views

No colored output in less for the ls command

If I run ls I get colored output which I find pretty handy for quickly getting a glance of the kind of file. When I try to pipe it to less even with the -r and -R flags the coloring always get lost. I ...
0
votes
3answers
66 views

Reuse output from command 1 in command 2 using a pipe

I've got a command in linux which returns a list of numbers. Now I need this numbers to form a directory path and cat the file located at that path. For example: myCommand returns: 1 10 30 40 And ...
1
vote
1answer
1k views

How to pipe scp/ssh file transfer(pull) through tar (remote->local transfer initiated from local machine)

I would like to pipe an scp pull operation(sample without tar below) through tar, so that only one tar file is written for about 10 directories: user@localhost:/home/user> scp -r ...
1
vote
1answer
56 views

Non-block poll from two pipes?

Here is what I want to do.. mkfifo infrequentlog frequentlog sort -m infrequentlog frequentlog > mylog funcA > infrequentlog funcB > frequentlog So that when monitor tail, I can see the ...
4
votes
2answers
491 views

How can I catch a command exit code for later, in a Makefile?

I have a makefile, with a test suite target, like so: rcheck: foo This shows the output from the foo command on stdout, and returns an exit code from foo which is handled by make as normal. ...
2
votes
1answer
172 views

Bash Named Pipe - Atomic Reading

I have set up one script to write to a named pipe, then four scripts to read the named pipe. For most of the entries, there are no problems, but when there is little processing to be done on the ...
1
vote
0answers
49 views

What to use instead of `buffer` for 1GB buffer?

buffer(1) seems to be old-ish and have hard-coded values preventing it to cache large amount of data. $ buffer -m 1G max_shmem 1 too low // it doesn't even understand gigabytes $ buffer -m 1000M ...
1
vote
1answer
48 views

pipe a file to a slow processing script question

Let's say I have a very large file and I want to process every line in that file by piping it to script. cat large_file.txt | python processor.py I'm not completely sure how the above operation ...
2
votes
1answer
52 views

pipe speed benchmark with machine readable output

I'm looking for a tool similar to pv which is producing machine readable output. And i cant believe that nobody did it bevore... I looked at the manpages of pv, pipemeter, dd and cpipe but all these ...
7
votes
4answers
193 views

Split an input for different command and combine the result

I know how to combine the result of different command paste -t',' <(commanda) <(commandb) I know pipe same input to different command cat myfile | tee >(commanda) >(commandb) Now ...
1
vote
1answer
151 views

Why does bash output data instead of executing, when a script is piped?

I'm running the following script on Ubuntu 14.04: #!/bin/bash apt-get purge -y nginx apt-get install -y nginx date When I run it like cat /tmp/script | bash, apt-get starts installing, then ...
20
votes
2answers
1k views

Pipes, how do data flow in a pipeline?

I don't understand how the data flows in the pipeline and hope someone could clarify what is going on there. I thought a pipeline of commands processes files (text, arrays of strings) in line by line ...
0
votes
2answers
145 views

ps output / pipeline execution order

I run ps aux | grep somethinghere. The output shows the grep somethinghere as a running process. My question is, shouldn't ps aux finish first, then grep somethinghere run on its output? Maybe there ...
4
votes
3answers
980 views

Piping a script with “read” to bash

I need to run a script by piping it through bash with wget(rather than running it directly with bash). $ wget -O - http://example.com/my-script.sh | bash It's not working because my script has read ...
1
vote
3answers
111 views

Tailing log files in two different hosts

I'm trying to tail two log files in different hosts and use the merged output as a single one. ssh HOST_1 "tail -f MY_LOG_FILE" I'm also using mkfifo to create a named pipe. mkfifo MY_PIPE The ...
3
votes
2answers
167 views

How to use tee to capture STDOUT from a code block to a filename defined within the block?

I want to send STDOUT for a script block to a file which name is defined by a variable within the block. However, when I use tee, it seems the variable outside the block doesn't exist anymore. Without ...
1
vote
1answer
71 views

Output Redirection

I am using the tee command to output the compilation errors of a program into a file along with the terminal. gcc hello.c | tee file.txt This is the command I have used. The compilation errors are ...
0
votes
1answer
59 views

exit status and no output

I have a command such that bar > /dev/null and I want to know the exit status of bar. I read some posts su about ${PIPESTATUS[0]} but this works when one pipes the output via | and I can't make ...
2
votes
2answers
150 views

What exactly is the difference between pipes and streams?

I read about pipes and streams and I’m still confused on how it’s implemented. A program is started and it reads data from “standard input” stream (stdin), which is where the keyboard sends data to. ...
0
votes
0answers
30 views

Pipe to `ls` Not Working as Expected [duplicate]

When I issue: [dave@dhost ~]$ which tar | ls -l I get the output of ls -l of the pwd. I expected to get: -rwxr-xr-x. 1 root root 395472 Feb 21 2013 /bin/tar Why doesn't the pipe work as I ...
4
votes
3answers
357 views

Using | pipe character from a $variable makes it treat as just another argument in bash; how to escape it?

I have a bash script like this export pipedargument="| sort -n" ls $pipedargument But it gives the error ls: |: No such file or directory ls: sort: No such file or directory It seems to be ...
2
votes
2answers
252 views

Feed source command with a pipe

Previously I used source command like this: source file_name But what I'm trying to do is this: echo something | source Which doesn't work.
2
votes
3answers
80 views

Pipe files downloaded with wget into a function

I'm downloading a bunch of images with wget and I want to rm the files if they are empty with a function to which I can pass the file. My question is how do I pipe the images to a function like this: ...
1
vote
1answer
136 views

preserving colors when piping yaourt through sort

Can sort preserve piped in colors? I'm trying to modify yaourt -Qu | sort so I get color output. Tried googling this but I couldn't find a general way nor a special way to do this for sort.
0
votes
1answer
42 views

Why doesn't this tee with process substitution produce the 1st and chosen lines? [duplicate]

What I want is printing the output the first line ( table head ) of ps aux and the grep result. After search, I come up with following. ps aux | tee >(head -1 > /dev/tty) | grep mongo But I ...
1
vote
1answer
29 views

How do I make an alias to substitute single word in a piped command? [duplicate]

I use aliases a lot but right now only for use cases like alias i='sudo apt-get install -y'. I often would like to add an alias in the following form: alias cmd='echo [something] >> ...
3
votes
3answers
164 views

Redirecting stdout to terminal and file without using a pipe?

I have code, that goes something like this: #!/bin/bash VAR=0 func() { VAR=$((VAR+1)) echo 'Logging information.' } func 2>&1 | tee 'log.txt' echo "Should be 1: ${VAR}" When ...
3
votes
2answers
222 views

Am I hitting a race condition in bash?

I have a script which compares the output of a command with the output of the same command as it was ran previously, it works most of the time, but every now and then it doesn't work as expected. ...
2
votes
3answers
712 views

source /dev/stdin doesn't work as expected

Let's start with simple tests that work for me to check that source /dev/stdin can be used at all. # echo -ne 'echo a\necho b\n' | source /dev/stdin a b Now I would like to source an actual ...
4
votes
1answer
146 views

cmd2 `cmd1` vs cmd1 | xargs cmd2

I'm trying to find a file with find and then pass the found path to sqlite3. I'm only dropped into the sqlite> prompt if my command is of the form: sqlite3 `find . -type f -iname "*.db" | head ...
0
votes
1answer
81 views

Force ping to write before interrupt

I am trying to get ping to continuously write some times to a file. When I run $ ping 10 10.1.10.28 | perl -ne '/time=(\d+\.\d+)/ && print "$1\n"' It returns one number every several ...
6
votes
4answers
619 views

Back to back Pipes into a Command

Is there a way to pipe back to back values into a command? e.g. If I can command.pl and want to pipe in response, then y to confirm (because it doesn't have a way to auto confirm)? I've tried echo ...
0
votes
1answer
223 views

Using tee or pipe to log console output will cause my program not to start [duplicate]

I'm running hostapd on my raspberry PI to let it function as an accesspoint. It's working fine with the following command: sudo hostapd /etc/hostapd/hostapd.conf This will start hostapd and log all ...
2
votes
1answer
158 views

'Tar' the result of a 'find', preserving the directory structure

I want to tar all the *.txt files that I get as the result of a find command, that exist in a directory having a tree structure like this: Directory_name dir1 file1.pdf file1.txt dir2 ...
3
votes
1answer
144 views

Problem with getting interactive input in while read loop

I was doing something like the following minimal example: #! /bin/sh # $1 = file containing list of files to sync # $2 = source dir ; $3 = target dir cat "$1" | while read f do cp -i "$2"/"$f" ...
2
votes
2answers
269 views

How to read from a named pipe after the writer process exits?

I want to use named pipes as asynchrnonous task queues (JMS-like, perhaps). For example, suppose a client sends some data that I wish to insert into a database, but I want the client to disconnect ...
2
votes
1answer
101 views

Problems with find, xargs and egrep

I this is what I what I'm trying to end up with (except working) find ./ -mindepth 1 -type f -mtime +60 -print0 | xargs -0 egrep -vZ 'vvv|iii' What am I doing wrong? $ ll total 0 -rw-rw-r-- 1 ...
3
votes
2answers
668 views

Less resource hungry alternative for piping `cat` into gzip for huge files

I have some files of which some are very large (like several GB), which I need to concatenate to one big file and then zip it, so something like this: cat file1 file2 file3 file4 | gzip > ...