Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

Questions tagged [pipe]

A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

1
vote
1answer
28 views

Capturing all output from piping input into a cli program

When I do something like: $ echo "print \"test\"" | python I want: Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more ...
-2
votes
1answer
21 views

Pipe output of two programs running in parallel to another program [duplicate]

I have two programs foo.sh and bar.sh both of which contain infite loops, produce continuous Output and should run in parallel. I would like to pipe the output of both those programs to a third ...
1
vote
1answer
45 views

how to pipe ls -ltr after list of files to capture date and time

we have script that print all bad wsp files ./print_bad_wsp_files.sh ./aaaa/rrr/aaaa/fff/ooo/min.wsp ./aaaa/rrr/aaaa/fff/ooo/p50.wsp ./aaaa/rrr/aaaa/fff/ooo/min.wsp ./aaaa/rrr/aaaa/fff/ooo/p50.wsp ...
2
votes
2answers
53 views

Using pipe STDOUT as a variable?

Fairly new to Bash, about a grueling week. I love it so far and really like long chaining pipes. What I'm noticing though is that I have to break the pipe if I need to use the STDOUT as a variable. ...
5
votes
0answers
41 views

What probability distribution models this race condition?

Consider the following command: bash -c "echo x; cat 1" | tee 1. My understanding is that it would fork to a new shell, write x to stdout, write file 1 not found to stderr, exit and return control ...
4
votes
3answers
142 views

Pipe filenames to zmv

I find it much easier to use find and then grep -v in the pipe to filter files instead of developing complex regex patterns. However, when I pipe something to zmv like this: find | grep -v TFLM | zmv ...
0
votes
1answer
39 views

multiple pipes in loop, saving pipeline-result to array

I am trying to do the following (using bash): Search for files that always have the same name and extract data from these files. I want to store the extracted data in new arrays I am almost there, I ...
0
votes
1answer
25 views

Why does redirecting output to a file produce a table?

I would like to setup a cron job to periodically record a measurement. A USB memory stick is installed in a router: user@router:/tmp/home/root# uname -a Linux router 2.6.36.4brcmarm #2 SMP ...
1
vote
1answer
20 views

Waiting for output: unexpected behaviour when piping xinput test to grep

After % xkbset -repeatkeys % xinput test 14 | grep -m 1 "key release 36" and holding the second return, nothing happens, as expected. After releasing the second return, I get key release 36 Now I ...
0
votes
1answer
39 views

AWK in the middle of a pipe returns no output

I want to process all changed logfiles from apache and made a simple oneliner: inotifywait --format="%w%f" -rm /var/log/apache2/ | gawk '{ print }' This is exactly my start. But if I pipe that ...
0
votes
1answer
28 views

Unexpected output of xargs

The statement below should print afc abc but it prints abc abc instead I tried many times but every time same result Don't know why this happens could anyone explain and help me? bash echo abc|xargs -...
2
votes
1answer
29 views

Getting error code of the first command in pipe in ash

I use wget to download a big file to two FIFOs like that: wget <wget-args> -O - | tee -a "$fifo1" >> "$fifo2" I want to retrieve the error code of wget on failure. set -o pipefail ...
5
votes
1answer
505 views

What's the difference in these commands - cat piped to dd and just cat? [duplicate]

I'm curious to know the difference between these two commands in Linux: $ cat ./boot.bin ./kernel.bin /dev/zero | dd bs=512 count=2880 of=devos.img and $ cat ./boot.bin ./kernel.bin > devos.img
1
vote
1answer
36 views

AIX SSH with Pipe operation one liners fail

I have a one liner to collect the errpt information on each AIX VIO server using ssh. But the below command is not working. I'm piping the command to execute on root shell on VIO server (oem_setup_env)...
0
votes
0answers
29 views

Python command history when piping into tee

I want to use tee with python and still be able to use command history when in interactive mode. So whitout tee I can do the following: $ python Python 3.6.5 (default, Apr 30 2018, 16:04:33) [GCC 7....
10
votes
2answers
435 views

How (and why) use stderr for both reading and writing?

According to this answer by schily, less reads navigation commands from stderr if it's not able to open /dev/tty. This seems puzzling, since I've never seen anything write to another program's stderr ...
-2
votes
2answers
45 views

variable assignment not working in pipe

why doesn't value get assigned to variable when it's piped. for example: a=15|echo $a Trying to group command's with {} and pass it to pipe, but value initialized in {} can't be used by command ...
3
votes
1answer
40 views

How can a pipe collapse?

I've just read this answer on DBA about moving databases. It contains this sentence: The only problem is when the database is too big and the pipe collapses. which is about this pipeline: ...
3
votes
1answer
46 views

Why does `cd` have no effect if output is piped?

I accidentally typed cd /tmp | tail which is of course not what I meant. But to my surprise, I then did not change directory. Why is this? For comparison cd /tmp > /tmp/foo does result in a ...
35
votes
3answers
3k views

How do pipelines limit memory usage?

Brian Kernighan explains in this video the early Bell Labs attraction to small languages/programs being based on memory limitations A big machine would be 64 k-bytes--K, not M or G--and so that ...
7
votes
5answers
338 views

sed only removing \n provided by sed '=' command

I created a file using: printf 'this is \n not is \n is is \n this biz' > file2 when I try to remove all \n(newline) it only remove sed's own inserted number's newline sed '=' file2 | sed 'N; s/...
0
votes
2answers
87 views

Pipes in bash script working while the same script fails in crontab

I encountered a strange behavior (to me). I have written a larger script which is working well. When I try to start the script from a crontab-defined job, the following lines are causing issues: ...
0
votes
1answer
40 views

File system for Pipe Device

From Chapter 5 of the book The design of Unix operating system by Bach, an algorithm to create an (unnamed) pipe is given. It states that, assign an inode from pipe device (algorithm ialloc) A pipe ...
0
votes
2answers
42 views

Reading a certain number of bytes from standard input and then closing the pipe

I’m trying to read some bytes out of /dev/urandom, keep only the ones I can type easily, and trim the result to 30 characters. I can’t figure out how to get the “only 30 characters” behavior when the ...
-1
votes
1answer
17 views

How can i send for 1 minute via a pipe, random numbers of 2 or 3 figures from the parent process to the child one?

Additionally to this task I have to count in the child process all the numbers received from the parent process and print them! Thanks in advance! :)
-1
votes
1answer
56 views

I'm trying to open an image in the terminal without any success

I want to open an image from a specific date and time inside the terminal. I located the image with the following command: # find . -type f -name '*2018-05-24*' | grep 18-46-31 ./Screenshot from 2018-...
1
vote
2answers
146 views

xargs + chmod - file or directory not found

I am trying to recursively change the permission of all files and directories in my project. I found a post on the magento forum saying that I can use these commands: find ./ -type f | xargs chmod ...
2
votes
2answers
52 views

Can I pipe output from a started process? [duplicate]

Say I've started program and it continues writing some output to stdout. Is there some way to add a pipe so that I can, for example, count the number of lines it has produced with wc -l?
-2
votes
1answer
77 views

How can we make `time` apply to a pipeline or its component? [duplicate]

time command1 | command2 Does time apply to command1 or command1 | command2? If your answer is one of the two, how would you specify the other? Could you explain your answer based on the grammar ...
3
votes
1answer
32 views

Receive multiple files from pipe and write to different destinations

I have this command locally: nc localhost 3440 | tar -x -O > ores.tgz if I receive one file from the server, it will be written to ores.tgz in my pwd. However, if I receive multiple files from ...
0
votes
1answer
47 views

difference between 2>&1 | tee output.log and |& tee output.log

I wanted to know the difference between the following two commands 2>&1 | tee output.log and |& tee output.log Looking for pipes to save command's output I've reached this explanation. ...
2
votes
3answers
201 views

How to echo a substring from the output of a pipe?

I found here how to extract a substring in bash, but I'm lost as to how to apply this after a pipe. For instance: some func | echo ${string:12:5} How do I assign the output of some func to the ...
0
votes
0answers
19 views

How can i pipe from my shell script to aircrack-ng

how can i pipe from my shellscript to aircrack-ng, small example: nano ./tool1.sh **Content inside tool1.sh(Shell Script)** -> var="abcdefghi"; tool1.sh | aircrack-ng -w - -e MyWireless file.cap ...
3
votes
2answers
472 views

Command with pipeline and redirection

What is the order of commands executed which have both pipeline and output redirection in it? Say we do the following: Charles@myzone:/tmp$ mkdir /tmp/testdir Charles@myzone:/tmp$ cd /tmp/...
0
votes
1answer
27 views

How is stdout piped in to a file located in a different directory?

I want to pipe the stdout of my python file exploit1.py to be accepted as a command line argument of a C executable lab2C.c. My intention is to overflow the char buffer in lab2C.c so that the memory ...
0
votes
0answers
20 views

SIGINT received by pipeline consumer is ignored by default?

I have a pipeline like so: foo | bar if I issue a SIGINT in terminal (in the form of ctrl+c), the second process (bar) will receive the SIGINT, but it just seems to ignore it, almost certainly ...
3
votes
1answer
311 views

Piping output of a command into grep and then into another command [duplicate]

I am taking output of one script, then piping this putput to grep and piping the output of grep into another script: ./script1 | grep 'expr' | ./script2 However, the second script never gets any ...
4
votes
1answer
47 views

Determine if producer is outpacing consumer in pipeline

If I have: node foo.js | node bar.js is there a way to determine if there is a queue in between them that's building up? in other words if the producer is outpacing the consumer w.r.t. stdio?
3
votes
2answers
63 views

grep PATTERN file, gzip when pattern exists else do nothing

Setup: Linux GNU bash, version 4.3 if grep -c PATTERN $sourcefile then grep PATTERN $sourcefile | gzip > compressedfile.gz fi I want to prevent having to access the sourcefile twice. How ...
0
votes
2answers
51 views

The difference between * and pipe

Saying that I want to check the size of each file in some directory. Here is what I do: du -sh * Also, I can do: ls | xargs du -sh The two commands do exactly the same thing. I want to know if ...
0
votes
0answers
41 views

How to make pipeline fail immediately if any of its stages fail

Suppose I have a pipeline reader | processor | writer. Seems that by default if processor or writer fail in the best case pipeline will fail only after reader will try to send data to processor. I ...
0
votes
1answer
35 views

Programmatically closing pipeline / exiting [closed]

I have a pipeline like so: tail -n0 -f "${my_input}" | ql_receiver_lock_holder | while read line; do echo "$line" >> "${my_output}"; # xxx: how can I programmatically close the ...
2
votes
3answers
37 views

First data written in pipeline (is it consistent)

If I have this pipeline: echo "bar" | nc localhost 6969 < <(echo "foo") will "foo" always be written to the tcp socket before "bar"? Is there a precedence there that is consistent?
1
vote
1answer
94 views

Read and write to same netcat tcp connection

I have this pipeline: ( tail -f ${named_pipe} | nc localhost ${port} | ql_conditional_release &> "$HOME/debug.log" & disown; ) &> /dev/null so I am reading from the netcat ...
0
votes
1answer
25 views

Reference the stdin sent from piped sender process

Say I have this: delete_lock () { if grep -q 'PATTERN'; then # some command here fi cat >/dev/null } node foo.js | delete_lock say the node.js process writes "foodog" to ...
1
vote
2answers
119 views

Render mixture plain text and HTML output from a command in a terminal

I am using SDcv, Collins Cobuild 5 is it's dictionary, which output html, like: # czxyl @ archlinux in ~/Dropbox [12:57:27] C:16 $ sdcv hello Found 7 items, similar to hello. --&...
0
votes
1answer
49 views

Bash Pipeline Stops Working

I am writing a bash script to process output from a mosquitto_sub call: function testPipe(){ read foo IFS=' ' read -ra ARRAY <<< "$foo" topic=${ARRAY[0]} cmd=${ARRAY[1]} echo "topic = $...
1
vote
1answer
51 views

socat and gawk piped

I'm trying to manipulate a TCP stream using gawk. The TCP server streams out a text string separated by whitespaces. I want to add an extra string at the beginning and then separate the remaining ...
0
votes
2answers
99 views

Prepend string before each line of stdin

I have this script which sends logs from each Kubernetes pod to stdout: #!/usr/bin/env bash set -e; oc project cdt-dev pods=$(oc get pods --show-all=false -o name) for j in ${pods}; do oc ...
1
vote
1answer
108 views

How does a temporary file differs from a pipe? [duplicate]

When one sends data (say stdout) to a temporary file, it is likely that something would be done with that data (the temporary file will be used as stdin for some process), then the temporary file will ...