Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [pipe]

A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

6
votes
0answers
56 views

Why does the rpm command have a '--pipe' option?

From rpm's man page: --pipe CMD Pipes the output of rpm to the command CMD. Why was this added? Why would you use it instead piping via the shell itself?
0
votes
1answer
9 views

concatenating 2 commands with tar (wc -l + echo ok || echo fail)

I want to do 2 things with a tar.gz file: Check for failure If successful - check number of files extracted I found this for the 1st option. I now run my command like this: tar -xvzf bad_file.tar.gz ...
-1
votes
2answers
48 views

Tee does not write to file when combined to pipe

I'm trying to run a C code with matlab, developed by others. In c, it's written to stdout, then in matlab a pipe is created and read: unix('rm -f /tmp/matdata.in;mkfifo /tmp/matdata.in'); unix(['./',...
0
votes
2answers
28 views

How to use sed on all matching files?

I am trying to run sed on the output from find | grep \.ext$ However, sed requires input file and output file. How can I save the output line to a variable, or pass it to sed?
-1
votes
2answers
33 views

gunzip on all present files safety regarding memory

I want to look for the presence of a string on all gziped files on server. I am afraid if I run something like gunzip -c *.gz | grep "string to be searched" server get overloaded with cache for ...
18
votes
1answer
2k views

Why does piping `tar` into `dd` not stop until the disk is full?

I have a tar archive of a single disk image. The image inside this tar file is about 4GB in size. I pipe the output of tar xf into dd to write the disk image to an SD card. The diskdump never stops ...
1
vote
1answer
32 views

Pipe Output of Executable to Existing Netcat Connection

I open a terminal window and connect to shell using netcat: Terminal Window 1: nc hostname port Then, I navigate to a directory and open a file,... etc (I would need to be able to do this myself, ...
0
votes
0answers
38 views

pipe(), fork() and for loops [migrated]

I am writing a program for an assignment where I am supposed to create multiple child processes from a parent, and then continually pipe information to their standard inputs to run a text-based game (...
5
votes
3answers
648 views

Opening most recently modified file in vim

The following commands work $ ls -1t | head -1 git_sync_log20180924_00.txt $ vi git_sync_log20180924_00.txt But this does not $ ls -1t | head -1 | vi Vim: Warning: Input is not from a terminal Vim:...
0
votes
1answer
49 views

How to read only the last line of a live input stream in a while read loop

I have a very simple bash script that synchronizes data to a USB key as soon as new files appear. It is poorly written but it works, however I want to minimize CPU time so I used sleep but since I use ...
4
votes
3answers
88 views

In what format does piped output get sent and received from one command/program to another? [duplicate]

When a command is piped to another command in what way or in what format does the piped output exist/get sent? Is it a temporary file? Is it a string? And how does the command that receives the piped ...
2
votes
2answers
79 views

How to check left command's exit status in pipe “left | right” in dash when left command produces large output

It seems that the variable "PIPESTATUS" is unavailable in dash. Simple separate execution is not working because the left command produces very large output. I used a fifo to do this task: #!/bin/...
-1
votes
3answers
102 views

How does a program know what to read in a pipeline? [duplicate]

How does the unix command line program know what file to read? For example: cat someFile | foo How does the program foo know which file to read, and which process is responsible for opening and ...
1
vote
2answers
33 views

Print netcat input on terminal

I'm trying to write a script where I cat the content of a file and pipe it into nc: $cat somefile.txt | nc server port The file content is sent to the server as expected but on the terminal I only ...
4
votes
0answers
34 views

Random errors thrown by Bash shell: cannot open named pipe

I have found that keeping a detailed log of every command I run, from which path I ran it from and which user I sudo'd from, is extremely valuable as the years go by and I'm trying to remember what I ...
3
votes
1answer
72 views

Is there a way to specify a section of a pipeline be completely “pass through”?

I have a script in which data is processed by streaming it through a fairly large pipeline. Several sections of the pipeline are actually "switchboard" functions that do different things based on some ...
0
votes
2answers
36 views

Have xargs pass a flag from stdin instead of command parameter

Given an application, myApplication, that streams lines to stdout $ myApplication flag1 flag2 flag3 How do I use xargs to pass those values as flags to a command instead of passing them as ...
1
vote
2answers
75 views

Streaming Data to REST Service Using Unix Pipes

Based on an answer to another question I am using curl to stream the stdout of one process as the entity of a POST request: myDataGeneratingApp \ | curl -H "Content-Type: application/json" -H "...
1
vote
1answer
31 views

How to ignore interrupts with piped commands

In the following chain of piped commands, when an interrupt is sent with Ctrl-C, ping is able to print its summary statistics before exiting, as long as tee has the -i (ignore interrupts) flag: ping -...
20
votes
3answers
2k views

grep doesn't output until EOF if piped through cat

Given this minimal example ( echo "LINE 1" ; sleep 1 ; echo "LINE 2" ; ) it outputs LINE 1 and then, after one second, outputs LINE 2, as expected. If we pipe this to grep LINE ( echo "LINE 1" ; ...
0
votes
2answers
34 views

How does the f argument work in this example of the cut command?

I'm studying for the LPIC-1 exam and I'm stuck understanding the following example of the command cut: ifconfig enp3s0f2 produces the following result: enp3s0f2: flags=4099<UP,BROADCAST,MULTICAST&...
4
votes
1answer
26 views

piping a man page on MacOS into an editor

Using bash on Mac OS to edit some man pages for my own use. In Mac OS the command open -t filename will open the specified file in the system's default text editor. $ man somepage | col -b will ...
6
votes
1answer
386 views

cat and pipe vs. redirection [duplicate]

What's the difference between these two commands? cat /proc/uptime | awk '{print $1}' < /proc/uptime awk '{print $1}' Specifically, how does the second command work? Doesn't the redirection ...
2
votes
2answers
196 views

`watch` command with piping `|` [duplicate]

I want to keep monitoring a specific job on a slurm worload like cluster. I tried to use the watch command and grep the specific id. If the job id is4138, I tried $> watch squeue -u mnyber004 | ...
2
votes
6answers
79 views

How can I operate on \1 in sed (or $1 in awk)?

I have file with several lines. Each with a timestamp followed by text: 1534888050 some text 1534888051 some more text 1534888052 text ... and want to convert the timestamp to a readable format, ...
1
vote
1answer
32 views

Capturing all output from piping input into a cli program

When I do something like: $ echo "print \"test\"" | python I want: Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more ...
-2
votes
1answer
26 views

Pipe output of two programs running in parallel to another program [duplicate]

I have two programs foo.sh and bar.sh both of which contain infite loops, produce continuous Output and should run in parallel. I would like to pipe the output of both those programs to a third ...
0
votes
1answer
59 views

how to pipe ls -ltr after list of files to capture date and time

we have script that print all bad wsp files ./print_bad_wsp_files.sh ./aaaa/rrr/aaaa/fff/ooo/min.wsp ./aaaa/rrr/aaaa/fff/ooo/p50.wsp ./aaaa/rrr/aaaa/fff/ooo/min.wsp ./aaaa/rrr/aaaa/fff/ooo/p50.wsp ...
2
votes
2answers
94 views

Using pipe STDOUT as a variable?

Fairly new to Bash, about a grueling week. I love it so far and really like long chaining pipes. What I'm noticing though is that I have to break the pipe if I need to use the STDOUT as a variable. ...
9
votes
1answer
55 views

What probability distribution models this race condition?

Consider the following command: bash -c "echo x; cat 1" | tee 1. My understanding is that it would fork to a new shell, write x to stdout, write file 1 not found to stderr, exit and return control ...
4
votes
3answers
153 views

Pipe filenames to zmv

I find it much easier to use find and then grep -v in the pipe to filter files instead of developing complex regex patterns. However, when I pipe something to zmv like this: find | grep -v TFLM | zmv ...
0
votes
1answer
58 views

multiple pipes in loop, saving pipeline-result to array

I am trying to do the following (using bash): Search for files that always have the same name and extract data from these files. I want to store the extracted data in new arrays I am almost there, I ...
0
votes
1answer
30 views

Why does redirecting output to a file produce a table?

I would like to setup a cron job to periodically record a measurement. A USB memory stick is installed in a router: user@router:/tmp/home/root# uname -a Linux router 2.6.36.4brcmarm #2 SMP ...
1
vote
1answer
23 views

Waiting for output: unexpected behaviour when piping xinput test to grep

After % xkbset -repeatkeys % xinput test 14 | grep -m 1 "key release 36" and holding the second return, nothing happens, as expected. After releasing the second return, I get key release 36 Now I ...
0
votes
1answer
61 views

AWK in the middle of a pipe returns no output

I want to process all changed logfiles from apache and made a simple oneliner: inotifywait --format="%w%f" -rm /var/log/apache2/ | gawk '{ print }' This is exactly my start. But if I pipe that ...
0
votes
1answer
37 views

Unexpected output of xargs

The statement below should print afc abc but it prints abc abc instead I tried many times but every time same result Don't know why this happens could anyone explain and help me? bash echo abc|xargs -...
2
votes
1answer
52 views

Getting error code of the first command in pipe in ash

I use wget to download a big file to two FIFOs like that: wget <wget-args> -O - | tee -a "$fifo1" >> "$fifo2" I want to retrieve the error code of wget on failure. set -o pipefail ...
5
votes
1answer
534 views

What's the difference in these commands - cat piped to dd and just cat? [duplicate]

I'm curious to know the difference between these two commands in Linux: $ cat ./boot.bin ./kernel.bin /dev/zero | dd bs=512 count=2880 of=devos.img and $ cat ./boot.bin ./kernel.bin > devos.img
1
vote
1answer
42 views

AIX SSH with Pipe operation one liners fail

I have a one liner to collect the errpt information on each AIX VIO server using ssh. But the below command is not working. I'm piping the command to execute on root shell on VIO server (oem_setup_env)...
0
votes
0answers
43 views

Python command history when piping into tee

I want to use tee with python and still be able to use command history when in interactive mode. So whitout tee I can do the following: $ python Python 3.6.5 (default, Apr 30 2018, 16:04:33) [GCC 7....
10
votes
2answers
478 views

How (and why) use stderr for both reading and writing?

According to this answer by schily, less reads navigation commands from stderr if it's not able to open /dev/tty. This seems puzzling, since I've never seen anything write to another program's stderr ...
-2
votes
2answers
57 views

variable assignment not working in pipe

why doesn't value get assigned to variable when it's piped. for example: a=15|echo $a Trying to group command's with {} and pass it to pipe, but value initialized in {} can't be used by command ...
3
votes
1answer
42 views

How can a pipe collapse?

I've just read this answer on DBA about moving databases. It contains this sentence: The only problem is when the database is too big and the pipe collapses. which is about this pipeline: ...
3
votes
1answer
49 views

Why does `cd` have no effect if output is piped?

I accidentally typed cd /tmp | tail which is of course not what I meant. But to my surprise, I then did not change directory. Why is this? For comparison cd /tmp > /tmp/foo does result in a ...
35
votes
3answers
4k views

How do pipelines limit memory usage?

Brian Kernighan explains in this video the early Bell Labs attraction to small languages/programs being based on memory limitations A big machine would be 64 k-bytes--K, not M or G--and so that ...
7
votes
5answers
374 views

sed only removing \n provided by sed '=' command

I created a file using: printf 'this is \n not is \n is is \n this biz' > file2 when I try to remove all \n(newline) it only remove sed's own inserted number's newline sed '=' file2 | sed 'N; s/...
0
votes
2answers
100 views

Pipes in bash script working while the same script fails in crontab

I encountered a strange behavior (to me). I have written a larger script which is working well. When I try to start the script from a crontab-defined job, the following lines are causing issues: ...
0
votes
1answer
46 views

File system for Pipe Device

From Chapter 5 of the book The design of Unix operating system by Bach, an algorithm to create an (unnamed) pipe is given. It states that, assign an inode from pipe device (algorithm ialloc) A pipe ...
1
vote
2answers
48 views

Reading a certain number of bytes from standard input and then closing the pipe

I’m trying to read some bytes out of /dev/urandom, keep only the ones I can type easily, and trim the result to 30 characters. I can’t figure out how to get the “only 30 characters” behavior when the ...
-1
votes
1answer
17 views

How can i send for 1 minute via a pipe, random numbers of 2 or 3 figures from the parent process to the child one?

Additionally to this task I have to count in the child process all the numbers received from the parent process and print them! Thanks in advance! :)