Chains the standard streams of a series of commands

learn more… | top users | synonyms

2
votes
2answers
94 views

Why do we need two file descriptors when creating an unnamed pipe?

I've been reading about unnamed pipes and as I understood them they're implemented as a buffer in memory. When creating the pipe I need to pass an array of size two and it returns two pointers (file ...
3
votes
2answers
314 views

OpenSSL buffering problem

I have a stream that I would like to encrypt in real time, but the problem is that openssl encrypt only when it could read from stdin enough bytes, otherwise it waits. Look at those examples: $ ...
2
votes
1answer
485 views

broken pipe error with popen and JS ffi

I am using a ffi for nodejs which for the most part has nothing to do with this question which is really about understanding pipes better, but does offer some context function exec(cmd) { var ...
1
vote
3answers
74 views

Variable scope in multiple pipes

I was trying to do something following, find . -name "*.dat" | get the basename of file | move filename returned by first command to basename returned by second command To give a concrete example, ...
8
votes
4answers
4k views

Are FIFO, pipe & Unix domain socket the same thing in Linux kernel?

I heard that FIFOs are named pipes. And they have exactly the same semantics. On the other hand, I think Unix domain socket is quite similar to pipe (although I've never made use of it). So I wonder ...
7
votes
1answer
241 views

“w | tail” chops text at right margin (78 characters)

Summary: w | tail chops rows after 78th column. When I run the "w" command, it works as expected. However, when I pipe the output to "tail", it chops the output to 78 columns wide (truncates anything ...
5
votes
3answers
544 views

Can I pipe any two processes to each other?

In this page from The Design and Implementation of the 4.4BSD Operating System, it is said that: A major difference between pipes and sockets is that pipes require a common parent process to set ...
2
votes
1answer
2k views

redirection to file issues with nohup and pipe

For the following command: ssh -t esolve@hostname 'sudo nohup bash -c "ls > log 2>&1 &"' I always got error information: nohup: ignoring input and appending output to ...
2
votes
1answer
67 views

process continous output of synclient

I am using synclient to track the position of finger on the touchpad. I use the following command. synclient -m 100 | awk '{print $2,$3}' This command gives the 'x' and 'y' co-ordinate on the ...
4
votes
2answers
90 views

Who cuts off first symbol of process substituted file and for what reason?

$ type 1.sh #!/bin/bash -eu php <(echo 12) $ ./1.sh 2 $ type 2.sh #!/bin/bash -eu cat <(echo 12) $ ./2.sh 12 $ type 3.sh #!/bin/bash -eu echo 12 | php $ ./3.sh 12 $ type 4.sh #!/bin/bash -eu ...
3
votes
1answer
199 views

How can I copy a file between server using bsdtar?

I'm looking for the fastest way of copying a file between 2 linux servers using bsdtar. With normal tar, I would do something like: ssh root@remote 'tar -cz -C /my/path/ big_file.txt' | tar -zxv ...
0
votes
2answers
472 views

Redirecting stdin with stdout to file

So I have a program that takes in user input and outputs text based on the input. EDIT2: I want to create a script that runs a C executable and the script feeds the C program input from a file and ...
4
votes
4answers
929 views

Getting stdin from a named pipe

What I am trying to do is run python in a terminal window and redirect it's stdin from a named pipe. Then I write to the named pipe in another terminal and have that command execute on python. ...
4
votes
1answer
758 views

Pass colors from ls through pipe to awk 'print' statement

This is a follow-up to my question from yesterday, Show sum of file sizes in directory listing. Thanks to Zero Piraeus and a point in the right direction by Mauritz Hansen, I now have function ...
3
votes
3answers
512 views

Can't process stdout with pipe as it comes

I'm running tshark on a fifo, and the following is a bare example of a loop that prints the output of tshark as it comes: tshark -i $fifo | while read line; do echo $line done The problem ...
3
votes
2answers
182 views

How to duplicate a stream and process both parts in a streaming way?

Sometimes I want to insert something in a pipeline for reporting or some other secondary use. It might be as simple as wc -l, or a more complex beast like awk or even a python script. It'd be nice to ...
5
votes
3answers
2k views

Output to stdout and at the same time grep into a file

I have a script that outputs text to stdout. I want to see all this output in my terminal, and at the same time I want to filter some lines and save them in a file. Example: $ myscript Line A Line B ...
5
votes
1answer
955 views

background process pipe-input

if i want to display "aaa" on screen: (1)$: echo aaa | cat ... works OK (2)$: echo aaa | ( cat ) ... works OK (3)$: echo aaa | ( cat & ) ... NOT working ...
0
votes
3answers
401 views

pass piped variable to perl oneliner

if you have something like: find . -d -maxdepth 1 | wc -l | perl ... what ways do you have to tell perl to take this variable that's coming its way and substract 1? I try to substract 1 since find ...
3
votes
3answers
517 views

Does one end of a pipe have both read and write fd?

As far as I understood one end of a pipe has both read and write fd's and the other end also has read and write fd's. Thats why when we are writing using fd[1], we are closing the read end e.g. ...
3
votes
2answers
332 views

Read data from a pipe for a certain amount of time (in seconds)

Is there an easy way to create a pipe that only lasts for a given amount of wall time? I want to be able to take a "snapshot" of what's being written STDOUT for a period of time (rather than bytes or ...
0
votes
1answer
418 views

ps | grep shows bad output in subshell with jobs running in the background

I am using bash. I have a file named "a2draw" that contains only 1 line: sleep 99999 I start it using this command: bash a2draw & Now, I know and understand the trick with square bracket ...
1
vote
0answers
267 views

avconv/ffmpeg output through /dev/ttyAMA0

I tried to pipe streaming video from video4linux2 webcam connected to a Raspberry Pi through /dev/ttyAMA0 to a computer using an Arduino as a intermediary. First of all, is it possible? Because I was ...
27
votes
4answers
8k views

What are the advantages of using named pipe over unnamed pipe?

I was reviewing a set of interview questions that are asked from a unix admin; I found a topic called "named pipe". I googled the topic; to some extent I have been able to understand it :- named ...
4
votes
3answers
1k views

Races when piping two commands to a named pipe

I want to have one process reading from a named pipe that receives data from multiple sources: $ mkfifo /tmp/p But I can't figure out how to get it to work consistently. First Scenario - this ...
0
votes
1answer
267 views

Can someone provide an xargs example piping mysql query data into another command?

Can someone provide an example for the command xargs? I want to do a mysql query to return the ID field of a column then feed that result into xargs into another command say mysql query delete. How ...
1
vote
1answer
127 views

What rules govern the parent-child relationship of processes launched by shell across a pipe?

#!/bin/ksh # start_service: start the service my_server_executable 2>&1 | my_pipe_following_shell_script & exit 0 After I run the above start_service script from command line, it is ...
10
votes
1answer
1k views

What are guarantees for concurrent writes into a named pipe?

For example, I created a named pipe like the following: mknod myPipe p And I read from it from some process (for example, some server). For example purposes, I used tail: tail -f myPipe If ...
3
votes
4answers
3k views

Piping output to text file within a for loop

I'm trying to do the following within a for loop: Find files that satisfy a condition Echo the name of the files to a log file. Gzip the file. I can get the script to find the files and echo their ...
1
vote
1answer
79 views

Encrypting import's output with gpg at the same time

The import command saves screenshot in a file. Is there a way to encrypt that file at the same time using gpg, without saving the image file first? gpg accepts input from stdin but it seems that ...
7
votes
3answers
1k views

tee + cat: use an output several times and then concatenate results

If I call some command, for instance an echo I can use the results from that command in several other commands with tee. Example: echo "Hello world!" | tee >(command1) >(command2) ...
4
votes
0answers
76 views

Combing head and tail in a single call via pipe [duplicate]

On a regular basis, I am piping the output of some program to either head or tail. Now, suppose that I want to see the first AND last 10 lines of piped output, such that I could do something like ...
1
vote
2answers
2k views

Why does pipe not work with cat and locate?

I am trying to perform this: locate pg_type.h | cat But this command simply does nothing different than locate pg_type.h What should I change ? I want to perform cat pg_type.h wherever pg_type.h ...
2
votes
1answer
116 views

Stderr of piped shell scripts isn't always displayed

I piped my own shell scripts for some testing and accidentally noticed something strange. Namely, the stderr of those piped processes isn't always displayed on the screen. I simplified the scripts ...
2
votes
1answer
2k views

Bash vs ksh pipes

I am stuck with some problems with my scripts in ksh. FWIW the problem which I am unable to overcome is that when I use a structure such as this command | while read VAR1 do many.commands using ...
2
votes
3answers
530 views

How to stop `grep` from truncating pipe output

Is there any way to have an echo at the end of a pipe simply append to the current output rather than removing it all? For example, how can I keep the output for my cat and grep and add "END OF ...
0
votes
2answers
95 views

Piping commands together in unix

How do I pipe the output of one command to another in unix? For example, If I want to pipe the output of sort to tail, how would I do that?
1
vote
1answer
306 views

Managing the output streams of many subprocesses with deadlocks

I have a Python script that does more or less this current_tasks = TaskManager() MAXPROCS = 8 while len(outstanding_tasks) > 0: if len(current_tasks.running) < MAXPROCS: ...
6
votes
2answers
3k views

Pseudo files for temporary data

I often want to feed relatively short string data (could be several lines though) to commandline programs which accept only input from files (e.g. wdiff) in a repeated fashion. Sure I can create one ...
3
votes
4answers
1k views

How to pipe the stdout to a file if the current command is already being piped through gzip

So I have this command I need to run then disown: innobackupex-1.5.1 --user=root --password=**** --stream=tar ./ | gzip - > /data/myfile.tar.gz How do I pipe any output to a file? adding ...
3
votes
1answer
50 views

Emulating a file

I have certain application that opens a file with given path (presumably with fopen or derivative), reads the file, possibly writes to it and closes it. I have no control over this application other ...
3
votes
1answer
176 views

No output from inotifywait | awk

I'm attempting to use part of a one-liner found here: Script to monitor folder for new files? When I try the following procedure I get no output whatsoever and I cannot figure out why. In terminal ...
2
votes
1answer
250 views

How to copy the output of pacman-color into a file and to display the output with the colors in the terminal

Originally I used pacman, not pacman-color and I copied the output into a file and I displayed the output in the terminal with the following command : pacman -Syu | tee file.log Unfortunately, it ...
5
votes
2answers
1k views

Why does Subversion give a broken pipe error when piped into head?

If I do a svn log | head after the tenth line of output I get an error message: svn: Write error: Broken pipe What's going on here? I haven't seen any other command do this when used with head. ...
2
votes
4answers
153 views

store and execute redirection/piped command in script

How do you store, and later execute, a command with redirection and and a pipe? (And is it the pipe, or the redirection, or both, that's causing my problem?) I tried to print the urxvt version in a ...
9
votes
2answers
1k views

echo vs <<<, or Useless Use of echo in Bash Award?

By now the Useless Use of cat Award is very well known, and there's also a mention of a Useless Use of echo (not relevant for this question). I'm wondering if there should be a "Useless Use of echo in ...
4
votes
1answer
539 views

Ksh loses data after piping 16K bytes

I recently found that ksh may lose some data after printing more than 16K bytes to the stdout if it is blocked for a couple of seconds. This test.sh script prints out 257*64 (16448) bytes: ...
1
vote
0answers
253 views

Why does this pipeline get SIGTTIN?

When I run the command for i in *.log; do cat "$i" | sort; done | more, the pipeline gets a SIGTTIN signal and is stopped. bash$ for i in *.log; do cat "$i" | sort; done | more ### (one screenful of ...
2
votes
1answer
366 views

Make stdout and stderr be buffered by the same buffer

Is it possible to force glibc to use the same buffer for stdout and stderr? If I tee the both outputs: ./tests/tests 2>&1 | tee log I get mixed messages (i.e. messages are ordered inside ...
7
votes
1answer
879 views

How to copy from/to the tmux 'clipboard' with shell pipes?

When working in a shell environment I run fairly often into the need to copy 'intermediate pipe output' around (eg. from/to already running editors, to other shells, other machines, etc.). When in a ...