Chains the standard streams of a series of commands

learn more… | top users | synonyms

5
votes
2answers
184 views

Obtaining PID of command earlier in Pipeline

I'm writing a bash script to use inotifywait to monitor a directory and kick off actions when changes are detected. Something like: inotifywait -m ... | while read f; do something; done Since ...
2
votes
2answers
134 views

turn off buffering for `hping` in OpenBSD

It would appear that the output from hping is fully buffered when piped to perl for further line-by-line processing, so piping hping to perl doesn't work. hping --icmp-ts example.ca | perl -ne 'if ...
4
votes
4answers
278 views

how do I trim each line in bash?

I would like to remove all leading and trailing spaces and tabs from each line in an output. Is there a simple tool like trim I could pipe my output into?
2
votes
3answers
115 views

How can I tell echo/printf in Bash to swallow escape codes, based on a condition?

I have a Bash script that is being run interactively and as a cron job. When run interactively it outputs colored text on the terminal. However, when run in cron it doesn't have a terminal and ...
3
votes
4answers
122 views

saving output of another command

I need to select certain lines from the log file and also save them to a text file. I tried the following but none of them are working as expected. The file "todel.txt" shows 0 bytes. tail -f ...
2
votes
2answers
98 views

Pipe command to tail: when is the first command aborted?

In this example: $ for i in {1..3}; do sleep 1; echo $i; done | head -n 2 why is the first command (for loop) killed only just before the 3 is displayed ? I expected it to be killed right after ...
2
votes
1answer
75 views

Appending to same array in various loops, only last values remain Bash 4

I have something like this: FILES=() for i in *.map do FILES+=($i) done find /var/candy -name "chocolate_[0-9]" | while read snack do FILES+=($snack) done for file in ...
1
vote
1answer
111 views

`cat /dev/ttyACM` loses data when piped through netcat

I'm reading data from /dev/ttyACM0 using cat /dev/ttyACM0 on a Raspberry Pi. The data comes from a device, which sends lines in the form 15.00 1316013 15 where the first number is a value, the ...
2
votes
1answer
172 views

Linux nuisance: /dev/stdin doesn't work with sockets

Linux has this blitheringly annoying peculiarity that /dev/stdin won't work with sockets - it is hardcoded to return ENXIO. Try this: socat TCP-OPEN:localhost:1234 EXEC:cat\ /dev/stdin,nofork ...
2
votes
1answer
219 views

Piping Sox and FFMPEG together

I'm processing a variety of audio files in a bunch of different formats and I'd like to unify their format and configuration using FFMPEG and SoX. There are two steps to my process: Convert the ...
21
votes
5answers
1k views

Pipe to multiple files in the shell

I have an application which will produce a large amount of data which I do not wish to store onto the disk. The application mostly outputs data which I do not wish to use, but a set of useful ...
1
vote
1answer
69 views

Pipe input to another process one line at a time

I have a script which can take one file name at a time and I want to run it for many files in a folder. So, I want to pipe the output of: find . -iname '*.m' to my script but one line at a time. ...
2
votes
3answers
297 views

Running tcpdump, tee and scp

I'm trying to run the following: $ tcpdump -l -X -i eth1 -n tcp port <port> | tee capture.txt | \ scp capture.txt root@10.3.3.227:/home/checker/ tcpdump -l -X -i eth1 -n tcp port ...
13
votes
4answers
262 views

pipe, { list; } only works with some programs

Need explanations from power users for such unpredictable behaviour: ps -eF | { head -n 1;grep worker; } UID PID PPID C SZ RSS PSR STIME TTY TIME CMD root 441 2 0 ...
0
votes
1answer
120 views

What is the meaning of cut's -d and -f options

How can this pipeline be explained: cat /etc/passwd | cut -d : -f 9 2>&1 | sort | uniq 2>&1 | wc -l I'm learning Unix by myself and I don't really get it. What does cut -d : -f 7 ...
5
votes
2answers
169 views

Is there a line-by-line bash pipe?

I remember reading somewhere (then I forgot where) about a rarely used type of bash pipe that could redirect output line-by-line. In other words, rather than redirecting the output once at the end, ...
12
votes
3answers
316 views

How to display the number of lines output by a command in real time?

I'm using svn export as part of a packager script for my application, and it looks like this command, like many others, doesn't have any kind of progress bar. I have two choices at the moment: ...
1
vote
1answer
107 views

Piped Logs in Apache?

Can someone walk me though how the data flows? Need to understand how I can intercept the log data, make changes to that data in situ, then save to a file.
4
votes
3answers
245 views

Why can't certain programs (like readlink) take input from a pipe?

I have a symbolic link to a script in my $PATH whose file I wanted to edit. I forgot the filepath, so I tried to do the following: $ which my_script_link | readlink I expected that to output the ...
1
vote
1answer
80 views

Pipe not picking up stdout

I am running a java program from a OS X 10.8 bash terminal, and are trying to rederect the output it is producing. However, when either running this throug a pipe, or rederecting it to a file, the ...
4
votes
3answers
326 views

Split stdin by lines

I want to invoke a command for every line of the standard input, much like xargs, but the line should be passed as standard input, not as a command-line argument: cat some-file.txt | <magic> | ...
4
votes
4answers
173 views

Feed OpenOffice via STDIN

I have an open office spreadsheet document stored inside a bash variable. I want to do something like this: echo "$openOfficeDoc" | ooffice To feed open office via STDIN. But it doesn't work. ...
1
vote
1answer
72 views

Flush data to file frequently for long running command? [duplicate]

I have a command that processes data slowly. The command processes lines from a file and writes the results to the output file data.txt: my_command > data.txt The issue I have is that I'd like to ...
1
vote
2answers
108 views

How to tell the “last” command to read from STDIN?

SERVER:~ # zcat /var/log/wtmp-20130827.gz | last -f - last: -: No such file or directory SERVER:~ # Without uncompressing the wtmp file, how can I see the output of if it with the last command?
5
votes
2answers
217 views

Use keyboard-interactive authentication when piping ssh output to other command

I'd like to pipe the output of ssh to a different command. For example: ssh myserver cat remote-file | diff local-file - The problem is that myserver asks for a password, but I can't enter it. (For ...
3
votes
1answer
200 views

find and rysnc?

I want to be able to search for files over 14 days and over 10k and than rsync those found files to a destination. Is there a way to combine these two commands? find ./ -mtime +14 -size +10k rsync ...
1
vote
1answer
47 views

Pipeline each line of a file to a specific variable in a second command

I want to execute the following instruction: basex -bword=ENTRY consulta.xq But for every line of a SampleText file (plain text mainly). The SampleText contains the following lines: hello evening ...
3
votes
1answer
284 views

Piping search term (not filename) to grep

I want to pick out certain lines containing a given number from a file. The file I want to search is called os_clusters/piRNA_clusters.bed. awk '{if (a[$0]++ == 0) {split($0,b,"."); ...
5
votes
3answers
218 views

Can you pipe to a .bash_profile function?

I received a great function for highlighting files in Apple's finder using the command-line. It's basically a wrapper for osascript. I got it from Mac OS X: How to change the color label of files ...
4
votes
2answers
360 views

How to download an archive and extract it without saving the archive to disk?

I'd like to download, and extract an archive under a given directory. Here is how I've been doing it so far: wget http://downloads.mysql.com/source/dbt2-0.37.50.3.tar.gz tar zxf dbt2-0.37.50.3.tar.gz ...
2
votes
3answers
80 views

Expand placeholders with same-named environment vars in text file?

There are a lot of variations on this question, but I wanted to ask it here anyways because I am so close (!) to finding a solution on this one - I just need a little nudge. Scenario I have several ...
0
votes
3answers
3k views

What makes a Unix process die with Broken pipe?

Here are some options I thought of, not sure which is the right one. a) There was an I/O error reading from the pipe. b) The process writing to the other end of the pipe died with a failure. c) All ...
1
vote
1answer
58 views

strange behaviour of strings command [duplicate]

I am trying to capture mysql traffic and pass those traffic to strings command as follows: tcpdump -i any -s 0 -l -w - dst port 3306 | strings This is working as expected and printing all mysql ...
1
vote
2answers
125 views

How to run chrome from command line by using process substitution?

I am using the following command without success - $ google-chrome <(grep "simple" SimpleDoc.txt) If I simply do a grep, I get - $ grep "simple" SimpleDoc.txt Very simple doc that contains ...
2
votes
2answers
399 views

Why does a named pipe not get deleted after system restart?

As far as I understand, named pipes are not written to disk but are stored in memory. Here's how I created a named pipe - $ mkfifo pipe21 $ grep "simple" SimpleDoc.txt > pipe21 & [1] 2775 $ ...
2
votes
4answers
650 views

Pipe the output of a command if it is successful

INPUT_FILE=`ls -rt $MY_DIR/FILE.*.xml | head -1 | xargs basename` I wanted to execute the second command (head -1) only if the first command is successful. How do I improve this command?
5
votes
2answers
186 views

Reusing pipe data for different commands

I would like to use the same pipe for different applications, like in: cat my_file | { cmd1 cmd2 cmd3 } Cmd1 should consume part of the input. Cmd2 should consume another part and so on. ...
3
votes
3answers
143 views

How to send part of the data through the pipe/file and later interactively?

I have a command I want to run, let's call it program. It is an interactive program similar to bash or python REPL. I want to start this program with some predefined input. I tried program << ...
1
vote
1answer
187 views

pipe curl colored output

this is the script color #!/bin/bash ESC_SEQ="\x1b[" COL_RESET=$ESC_SEQ"39;49;00m" case $1 in red ) COL_START=$ESC_SEQ"31;01m" ;; green) COL_START=$ESC_SEQ"32;01m" ;; yellow ...
2
votes
1answer
96 views

How can I work around the program failing if there is *any* stdin?

For example, the following works fine: /usr/bin/program It produces some output, and gets to result. But if I invoke it like this: echo -n | /usr/bin/program or this echo -n | bash -c ...
4
votes
1answer
287 views

How can I use tee and logger -t “some tag:”?

I've seen in another post which shows how to use tee and logger as shown below. tee >(logger) <<< "System Load is OK : $Current_loadadv" >(logger) is bash syntax to create a file ...
3
votes
4answers
343 views

How can I split a pipe into 16 byte sequences?

I'm writing a bash script to automate the decryption of a certain file; however the file is encrypted in 16 byte blocks, rather than simply encoded in its entirety. Here's some psuedo code to briefly ...
4
votes
2answers
231 views

How to capture stdout of a running process redirected to /dev/null

I want to capture stdout of running process whose stdout is being redirected to /dev/null. I tried using system call fcntl(), however the moment I close fd "1", it was being allocated for other files ...
4
votes
2answers
258 views

Problem with pipes. Pipe terminates when reader done

I am on OSX, using bash, trying to make sense of pipes. I wish to let a program communicate in two directions with a bash shell. I want to set this up in such a way that this is always the same shell, ...
2
votes
2answers
313 views

Does named pipe modify the filesystem?

As title. If I create a named pipe and then read/write on it, is the filesystem where the named pipe resides affected? I.e. is the data buffered on the filesystem until read, or does it reside in ...
1
vote
1answer
100 views

What to do when result from pipe is not last argument in next command

The example below shows what I want to do, namely take the result from my last pipe and put it into the middle of the next command (where I wrote $RESULT_FROM_FIND_COMMAND). $ find . -name ...
1
vote
1answer
98 views

Bash pipe output to more

I am wondering about some bash environment setting here: is it possible to set stdout of bash as a pipe to /bin/more? You know, like using vi as editor for your commands it should be possible to avoid ...
2
votes
2answers
90 views

Why do we need two file descriptors when creating an unnamed pipe?

I've been reading about unnamed pipes and as I understood them they're implemented as a buffer in memory. When creating the pipe I need to pass an array of size two and it returns two pointers (file ...
3
votes
2answers
227 views

OpenSSL buffering problem

I have a stream that I would like to encrypt in real time, but the problem is that openssl encrypt only when it could read from stdin enough bytes, otherwise it waits. Look at those examples: $ ...
2
votes
1answer
366 views

broken pipe error with popen and JS ffi

I am using a ffi for nodejs which for the most part has nothing to do with this question which is really about understanding pipes better, but does offer some context function exec(cmd) { var ...