Chains the standard streams of a series of commands

learn more… | top users | synonyms

2
votes
3answers
562 views

Running tcpdump, tee and scp

I'm trying to run the following: $ tcpdump -l -X -i eth1 -n tcp port <port> | tee capture.txt | \ scp capture.txt root@10.3.3.227:/home/checker/ tcpdump -l -X -i eth1 -n tcp port ...
13
votes
4answers
277 views

pipe, { list; } only works with some programs

Need explanations from power users for such unpredictable behaviour: ps -eF | { head -n 1;grep worker; } UID PID PPID C SZ RSS PSR STIME TTY TIME CMD root 441 2 0 ...
0
votes
1answer
233 views

What is the meaning of cut's -d and -f options

How can this pipeline be explained: cat /etc/passwd | cut -d : -f 9 2>&1 | sort | uniq 2>&1 | wc -l I'm learning Unix by myself and I don't really get it. What does cut -d : -f 7 ...
5
votes
2answers
288 views

Is there a line-by-line bash pipe?

I remember reading somewhere (then I forgot where) about a rarely used type of bash pipe that could redirect output line-by-line. In other words, rather than redirecting the output once at the end, ...
12
votes
3answers
398 views

How to display the number of lines output by a command in real time?

I'm using svn export as part of a packager script for my application, and it looks like this command, like many others, doesn't have any kind of progress bar. I have two choices at the moment: ...
1
vote
1answer
129 views

Piped Logs in Apache?

Can someone walk me though how the data flows? Need to understand how I can intercept the log data, make changes to that data in situ, then save to a file.
5
votes
3answers
664 views

Why can't certain programs (like readlink) take input from a pipe?

I have a symbolic link to a script in my $PATH whose file I wanted to edit. I forgot the filepath, so I tried to do the following: $ which my_script_link | readlink I expected that to output the ...
1
vote
1answer
109 views

Pipe not picking up stdout

I am running a java program from a OS X 10.8 bash terminal, and are trying to rederect the output it is producing. However, when either running this throug a pipe, or rederecting it to a file, the ...
4
votes
3answers
564 views

Split stdin by lines

I want to invoke a command for every line of the standard input, much like xargs, but the line should be passed as standard input, not as a command-line argument: cat some-file.txt | <magic> | ...
6
votes
5answers
305 views

Is it possible to feed Open Office via STDIN?

I have an Open Office Spreadsheet document stored inside a bash variable. I want to do something like the following to feed Open Office via STDIN: echo "$openOfficeDoc" | ooffice But it doesn't ...
1
vote
1answer
148 views

Flush data to file frequently for long running command? [duplicate]

I have a command that processes data slowly. The command processes lines from a file and writes the results to the output file data.txt: my_command > data.txt The issue I have is that I'd like to ...
1
vote
2answers
184 views

How to tell the “last” command to read from STDIN?

SERVER:~ # zcat /var/log/wtmp-20130827.gz | last -f - last: -: No such file or directory SERVER:~ # Without uncompressing the wtmp file, how can I see the output of if it with the last command?
5
votes
2answers
262 views

Use keyboard-interactive authentication when piping ssh output to other command

I'd like to pipe the output of ssh to a different command. For example: ssh myserver cat remote-file | diff local-file - The problem is that myserver asks for a password, but I can't enter it. (For ...
3
votes
1answer
770 views

find and rysnc?

I want to be able to search for files over 14 days and over 10k and than rsync those found files to a destination. Is there a way to combine these two commands? find ./ -mtime +14 -size +10k rsync ...
1
vote
1answer
57 views

Pipeline each line of a file to a specific variable in a second command

I want to execute the following instruction: basex -bword=ENTRY consulta.xq But for every line of a SampleText file (plain text mainly). The SampleText contains the following lines: hello evening ...
5
votes
1answer
1k views

Piping search term (not filename) to grep

I want to pick out certain lines containing a given number from a file. The file I want to search is called os_clusters/piRNA_clusters.bed. awk '{if (a[$0]++ == 0) {split($0,b,"."); ...
5
votes
3answers
268 views

Can you pipe to a .bash_profile function?

I received a great function for highlighting files in Apple's finder using the command-line. It's basically a wrapper for osascript. I got it from Mac OS X: How to change the color label of files ...
5
votes
2answers
1k views

How to download an archive and extract it without saving the archive to disk?

I'd like to download, and extract an archive under a given directory. Here is how I've been doing it so far: wget http://downloads.mysql.com/source/dbt2-0.37.50.3.tar.gz tar zxf dbt2-0.37.50.3.tar.gz ...
2
votes
3answers
107 views

Expand placeholders with same-named environment vars in text file?

There are a lot of variations on this question, but I wanted to ask it here anyways because I am so close (!) to finding a solution on this one - I just need a little nudge. Scenario I have several ...
2
votes
3answers
9k views

What makes a Unix process die with Broken pipe?

Here are some options I thought of, not sure which is the right one. a) There was an I/O error reading from the pipe. b) The process writing to the other end of the pipe died with a failure. c) All ...
1
vote
1answer
70 views

strange behaviour of strings command [duplicate]

I am trying to capture mysql traffic and pass those traffic to strings command as follows: tcpdump -i any -s 0 -l -w - dst port 3306 | strings This is working as expected and printing all mysql ...
1
vote
2answers
203 views

How to run chrome from command line by using process substitution?

I am using the following command without success - $ google-chrome <(grep "simple" SimpleDoc.txt) If I simply do a grep, I get - $ grep "simple" SimpleDoc.txt Very simple doc that contains ...
2
votes
2answers
980 views

Why does a named pipe not get deleted after system restart?

As far as I understand, named pipes are not written to disk but are stored in memory. Here's how I created a named pipe - $ mkfifo pipe21 $ grep "simple" SimpleDoc.txt > pipe21 & [1] 2775 $ ...
4
votes
4answers
2k views

Pipe the output of a command if it is successful

INPUT_FILE=`ls -rt $MY_DIR/FILE.*.xml | head -1 | xargs basename` I wanted to execute the second command (head -1) only if the first command is successful. How do I improve this command?
5
votes
2answers
285 views

Reusing pipe data for different commands

I would like to use the same pipe for different applications, like in: cat my_file | { cmd1 cmd2 cmd3 } Cmd1 should consume part of the input. Cmd2 should consume another part and so on. ...
3
votes
3answers
180 views

How to send part of the data through the pipe/file and later interactively?

I have a command I want to run, let's call it program. It is an interactive program similar to bash or python REPL. I want to start this program with some predefined input. I tried program << ...
1
vote
1answer
317 views

pipe curl colored output

this is the script color #!/bin/bash ESC_SEQ="\x1b[" COL_RESET=$ESC_SEQ"39;49;00m" case $1 in red ) COL_START=$ESC_SEQ"31;01m" ;; green) COL_START=$ESC_SEQ"32;01m" ;; yellow ...
2
votes
1answer
131 views

How can I work around the program failing if there is *any* stdin?

For example, the following works fine: /usr/bin/program It produces some output, and gets to result. But if I invoke it like this: echo -n | /usr/bin/program or this echo -n | bash -c ...
4
votes
1answer
545 views

How can I use tee and logger -t “some tag:”?

I've seen in another post which shows how to use tee and logger as shown below. tee >(logger) <<< "System Load is OK : $Current_loadadv" >(logger) is bash syntax to create a file ...
3
votes
5answers
428 views

How can I split a pipe into 16 byte sequences?

I'm writing a bash script to automate the decryption of a certain file; however the file is encrypted in 16 byte blocks, rather than simply encoded in its entirety. Here's some psuedo code to briefly ...
4
votes
3answers
486 views

How to capture stdout of a running process redirected to /dev/null

I want to capture stdout of running process whose stdout is being redirected to /dev/null. I tried using system call fcntl(), however the moment I close fd "1", it was being allocated for other files ...
4
votes
2answers
673 views

Problem with pipes. Pipe terminates when reader done

I am on OSX, using bash, trying to make sense of pipes. I wish to let a program communicate in two directions with a bash shell. I want to set this up in such a way that this is always the same shell, ...
2
votes
2answers
600 views

Does named pipe modify the filesystem?

As title. If I create a named pipe and then read/write on it, is the filesystem where the named pipe resides affected? I.e. is the data buffered on the filesystem until read, or does it reside in ...
0
votes
1answer
152 views

What to do when result from pipe is not last argument in next command

The example below shows what I want to do, namely take the result from my last pipe and put it into the middle of the next command (where I wrote $RESULT_FROM_FIND_COMMAND). $ find . -name ...
1
vote
1answer
253 views

Bash pipe output to more

I am wondering about some bash environment setting here: is it possible to set stdout of bash as a pipe to /bin/more? You know, like using vi as editor for your commands it should be possible to avoid ...
2
votes
2answers
98 views

Why do we need two file descriptors when creating an unnamed pipe?

I've been reading about unnamed pipes and as I understood them they're implemented as a buffer in memory. When creating the pipe I need to pass an array of size two and it returns two pointers (file ...
3
votes
2answers
361 views

OpenSSL buffering problem

I have a stream that I would like to encrypt in real time, but the problem is that openssl encrypt only when it could read from stdin enough bytes, otherwise it waits. Look at those examples: $ ...
2
votes
1answer
537 views

broken pipe error with popen and JS ffi

I am using a ffi for nodejs which for the most part has nothing to do with this question which is really about understanding pipes better, but does offer some context function exec(cmd) { var ...
1
vote
3answers
75 views

Variable scope in multiple pipes

I was trying to do something following, find . -name "*.dat" | get the basename of file | move filename returned by first command to basename returned by second command To give a concrete example, ...
9
votes
4answers
5k views

Are FIFO, pipe & Unix domain socket the same thing in Linux kernel?

I heard that FIFOs are named pipes. And they have exactly the same semantics. On the other hand, I think Unix domain socket is quite similar to pipe (although I've never made use of it). So I wonder ...
7
votes
1answer
249 views

“w | tail” chops text at right margin (78 characters)

Summary: w | tail chops rows after 78th column. When I run the "w" command, it works as expected. However, when I pipe the output to "tail", it chops the output to 78 columns wide (truncates anything ...
5
votes
3answers
622 views

Can I pipe any two processes to each other?

In this page from The Design and Implementation of the 4.4BSD Operating System, it is said that: A major difference between pipes and sockets is that pipes require a common parent process to set ...
2
votes
1answer
2k views

redirection to file issues with nohup and pipe

For the following command: ssh -t esolve@hostname 'sudo nohup bash -c "ls > log 2>&1 &"' I always got error information: nohup: ignoring input and appending output to ...
2
votes
1answer
68 views

process continous output of synclient

I am using synclient to track the position of finger on the touchpad. I use the following command. synclient -m 100 | awk '{print $2,$3}' This command gives the 'x' and 'y' co-ordinate on the ...
4
votes
2answers
94 views

Who cuts off first symbol of process substituted file and for what reason?

$ type 1.sh #!/bin/bash -eu php <(echo 12) $ ./1.sh 2 $ type 2.sh #!/bin/bash -eu cat <(echo 12) $ ./2.sh 12 $ type 3.sh #!/bin/bash -eu echo 12 | php $ ./3.sh 12 $ type 4.sh #!/bin/bash -eu ...
3
votes
1answer
218 views

How can I copy a file between server using bsdtar?

I'm looking for the fastest way of copying a file between 2 linux servers using bsdtar. With normal tar, I would do something like: ssh root@remote 'tar -cz -C /my/path/ big_file.txt' | tar -zxv ...
0
votes
2answers
532 views

Redirecting stdin with stdout to file

So I have a program that takes in user input and outputs text based on the input. EDIT2: I want to create a script that runs a C executable and the script feeds the C program input from a file and ...
4
votes
4answers
1k views

Getting stdin from a named pipe

What I am trying to do is run python in a terminal window and redirect it's stdin from a named pipe. Then I write to the named pipe in another terminal and have that command execute on python. ...
4
votes
1answer
884 views

Pass colors from ls through pipe to awk 'print' statement

This is a follow-up to my question from yesterday, Show sum of file sizes in directory listing. Thanks to Zero Piraeus and a point in the right direction by Mauritz Hansen, I now have function ...
3
votes
3answers
596 views

Can't process stdout with pipe as it comes

I'm running tshark on a fifo, and the following is a bare example of a loop that prints the output of tshark as it comes: tshark -i $fifo | while read line; do echo $line done The problem ...