Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [pipe]

A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in <unistd.h>. Shells provide pipe creation between processes using "|".

0
votes
2answers
22 views

Pipe and filter bash outputs in realtime

I am running some commands in bash (basically some ffmpeg commands), that I'm using grep and awk to filter out the results. The command takes some time and continously outputs some results as it ...
1
vote
0answers
37 views

Broken pipe error [on hold]

The following are my 2 Rscripts written as mapreduce code- 1) mapper.R #! usr/bin/env/ Rscript # mapper.R - Wordcount Program in R # script for Mapper (R-Hadoop Integration) trimWhiteSpace<- ...
15
votes
2answers
1k views

Understanding piped commands in Unix/Linux

I have two simple programs: A and B. A would run first, then B gets the “stdout” of A and uses it as its “stdin”. Assume I am using a GNU/Linux operating system and the simplest possible way to do ...
2
votes
1answer
58 views

Run multiple commands after pipe

I am piping streamer ouput to mpv player which works fine. streamer -o - | mpv - But before starting mpv , I want to kill one program and then pipe to mpv. streamer program takes some time to ...
2
votes
1answer
50 views

Why can't I chown a pipe? [closed]

From a discussion on useless use of cat. I decided to see if It was possible to avoid some of the troubles by changing the owner of the pipe. So I did: # cat | cat & [1] 16500 # cd /proc/16500/...
1
vote
1answer
36 views

Pipe stderr only if a terminal application crashes

I have a program which interacts with the user on the terminal. But if it crashes I'd like to pipe the stderr through another tool to tidy / prettyprint it. Is there a way to do this? As far as I ...
-1
votes
2answers
41 views

Only command after pipe executes, how does the pipe work in these scenarios?

What happens in the following two examples? Why is only the second command in each pipeline executed? How does the pipe work here? example 1 $ cat /var/log/messages | date Mon Apr 8 17:10:02 IST ...
3
votes
1answer
54 views

Make GNU Parallel not delay before executing arguments from STDIN

GNU Parallel, without any command line options, allows you to easily parallelize a command whose last argument is determined by a line of STDIN: $ seq 3 | parallel echo 2 1 3 Note that parallel does ...
2
votes
1answer
37 views

Cat blocking when piping output from /usr/bin/gssproxy (`timeout -k 1 1 /usr/bin/gssproxy | cat` does not terminate)

I use gssproxy under Arch Linux, in particular Name : gssproxy Version : 0.8.0-1 I encountered the following behavior: When trying to pipe the output of /usr/bin/gssproxy like ...
1
vote
1answer
42 views

Why isn't it possible to read from `stdin` with `read` when piping a script to bash?

I'm not looking for work-arounds or solutions for the issue. I'm fine with it not working like that in bash. I just don't understand why it doesn't work. I'm looking for an in-depth answer why the ...
0
votes
0answers
31 views

Bandit0 'Broken Pipe Error' Problem

Tried to connect with ssh -p 2220 bandit0@bandit.labs.overthewire.org and after successfully entering the password, I'm met with this error: packet_write_wait: Connection to 176.9.9.172 port 2220: ...
0
votes
1answer
38 views

Read paths on stdin and spawn a new interactive shell for each line

Say I have the results of a long-running process, like this: $ find $HOME -perm 777 Now, I could get the results in a pager like this: $ find $HOME -perm 777 | less and then cd to the relevant ...
2
votes
2answers
28 views

Check which strings from list never appear in recursive search of directoy

I have a list of strings stringList, where I want to recursively search a directory and find all of the strings in stringList that did not appear in any search result. Right now, I have cat ...
3
votes
2answers
59 views

Output redirection is not working with top piped to grep

When I run this command in Linux: $ top -b -d 20 | grep "load average" -A 20 > top.log top.log is always empty. But when I run this command: $ top -b -d 20 | grep "load average" -A 20 | tee top....
1
vote
1answer
36 views

Bash: child process and pipelining

The purpose of my script is to count how many lines there are in a file. I know I could use wc for example but the purpose of this exercise is to understand processes and pipelines in Linux. The ...
0
votes
1answer
39 views

How to run shell command inside awk and redirect to another shell command?

I want to get particular string from the files downloaded using curl and redirect the downloaded file content to another shell program. Here is the code : curl "http://file-list-url" -s | awk 'BEGIN {...
2
votes
1answer
50 views

Why does echo “a” | wc -m return 2? [duplicate]

Why does echo "a" | wc -m echo 2? Other variations in which I try to reduce the number of spaces yield the same result.
-1
votes
2answers
53 views

Pipe each batch of xargs trough wc -l

So my task is to find the file with the most hardlinks in a directory. So far i have : find . -name "file*" | xargs -I{} -n 1 find . -samefile {} which gives me: ./hardlinkFIle245 ./...
4
votes
2answers
218 views

pipe commands inside find -exec?

Let's suppose I want to find all .txt files and search for some string. I would do: find ./ -type f -name "*.txt" -exec egrep -iH 'something' '{}' \; What if I want to do a more complex filtering, ...
0
votes
1answer
49 views

echo, pipe and command substitution

I want to substitute an echo command followed with | (pipe). For example, $(echo "echo 'hello' | cat") returns 'hello' | cat I expect this to behave like echo 'hello' | cat which returns hello ...
1
vote
2answers
33 views

Two named PIPEs (PIPE_in/PIPE_out) connected with `tail -f` | String sent to PIPE_in doesn't reach PIPE_out

1.Create named PIPEs, pipe_in and pipe_out by running: $ mkfifo pipe_in $ mkfifo pipe_out 2.Connect pipe_in to pipe_out: TERM0: $ tail -f pipe_in > pipe_out 3.Send string hello world! to ...
0
votes
1answer
34 views

Getting the IP address of a given interface [duplicate]

I need the IP (v4) address of a device I specify. The response will be later used in a script. This is what I have: $ IFACE=eth1 $ ifconfig $IFACE |grep "inet " | awk '{print $2}' 10.0.0.33 Is there ...
0
votes
2answers
74 views

How to directly compress CURL output as named archive entry?

I've hundred of text files I need to CURL from remote host. Currently I've a loop which download each file and then compress it as new .zip entry. Is it possible to directly pipe CURL's output to an ...
0
votes
0answers
26 views

C++ script on Spark Cluster throws exit status 132

I'm trying to run a c++ program on spark cluster by using the rdd.pipe() operation but the executors throw: java.lang.IllegalStateException: Subprocess exited with status 132. The spark jar runs ...
1
vote
0answers
29 views

Getting awk value and outputting to a file [duplicate]

I need to get a value from a log file which I am getting using awk at the moment. I can get the number printing on the console but when I try to put that value into a new file nothing happens. Here is ...
0
votes
0answers
74 views

How to write Docker logs to a file in real time (à la `tail -f`)

I docker that output logs stdout stderr, which can be viewed using: docker logs -f $LOGS_CONTAINER_ID I also added 'sed', which puts the container id on each line: docker logs -f $LOGS_CONTAINER_ID ...
0
votes
1answer
22 views

Using bash script to pipe breakpoints to debugger prompt

I am using bash to write a script to make it easier to set breakpoints. I am trying to see if I can use echo and pipe to send set breakpoints commands to the java debugger jdb. The command I have ...
2
votes
2answers
63 views

Redirecting from right to left

From old habits, I'm always redirecting "from left to right", eg. cat file | bar foo | bar I've noticed you can redirect "from right to left". However, is it possible to do this for the second form (...
2
votes
3answers
159 views

Piping Multiple Numbers into Sed

Is it possible to pipe a list of numbers into sed? The pipeline I currently have looks something like: grep -nP 'foo' file_full.txt | sort | awk -F'[:;\t]' '{print $1,$3,$9,$13}' The output of ...
0
votes
1answer
74 views

Socat. Bridge TCP - SERIAL PORT. With log

I'm trying to create a bridge between TCP server and a client connected thorough a serial port using socat. I emulate my TCP server with the following command socat tcp-listen:8888,reuseaddr - I ...
0
votes
1answer
25 views

How to prompt data on piped commands [duplicate]

I'm trying to use this command: mysqldump --opt --hex-blob \ -h source -p xxxx -u user1 \ source_db | \ mysql \ -h target -p yyyy -u user2 \ target_db So, how can I rewrite it to ...
2
votes
1answer
59 views

Reverse the order of file matching as arguments

I have a program that reads files in a certain order. For this example, they all have the same extension, txt {program_name} {input 1...} {output} I happen to name the output file name "a.txt" so it ...
6
votes
3answers
182 views

What characterizes a file in Linux/Unix?

What characterizes a file in Linux/Unix? A file can have many types: regular file, directory, symlink, device file, socket, pipe, fifo, and more that I miss. For example, a symlink: $ sudo file /...
2
votes
2answers
87 views

Grep Ignoring Pipe?

I'm trying to pipe the output of a grep search into the input of another grep. Such as: grep search_query * | grep -v but_not_this But the second grep is not using the output of the previous search. ...
0
votes
1answer
60 views

Is it possible to manage total threads for pipes of multi-threaded commands?

I am running several tools that have multi-threading capabilities. For example: tool1 --threads=4 | tool2 --threads=4 | tool3 --threads=4 In an ideal case, I assume this should utilize 12 (4*3) ...
2
votes
1answer
44 views

2 grep calls with pipe doesn't work as expected

I came up with the following command: grep 'rtl' reg_list.txt | grep -E -o -n "id_\w+" reg_list.txt The idea is that first grep lines which contains rtl content, then grep only the words which starts ...
0
votes
1answer
138 views

Why curl > /dev/null and curl -o /dev/null has a large performance difference?

I am using Intel Atom D525, an extremely unpowerful processor for my family router with Debian 9. I tried both curl http://dl.google.com/dl/android/aosp/sailfish-pq2a.190205.003-factory-164a7269.zip &...
1
vote
1answer
85 views

how was the pipe(2) system call working in “traditional” Unix

This is what perldoc -f syscall says: There's a problem with syscall(SYS_pipe()): it returns the file number of the read end of the pipe it creates, but there is no way ...
2
votes
2answers
56 views

Bash: join pipes without deadlock

I want to list a bunch of filenames via find, pipe them through a utility (let's call this util) which outputs a new name for each input name, and then rename each file from the old name to the new. ...
2
votes
2answers
159 views

Bash: How to kill eval if the process that receives its output terminates

I have a little ugly bash script on my Ubuntu machine that contains the lines: search_command="find -L $(printf "%q" "$search_folder") \( ! -regex '.*/\..*/..*' \) -mindepth 1 2> /dev/null" for i ...
24
votes
8answers
2k views

Is piping, shifting, or parameter expansion more efficient?

I'm trying to find the most efficient way to iterate through certain values that are a consistent number of values away from each other in a space separated list of words(I don't want to use an array)....
1
vote
1answer
60 views

Stop netcat as soon as grep matches something

I am facing a problem using netcat in a bash script. I would like to match a specific output after sending a command and continue the script execution as soon as possible (not waiting for a netcat ...
0
votes
0answers
91 views

Piping password multiple times to a command via the command line

I am running Ubuntu 18.04 with bash and I would like to pass a password to a command that runs with sudo, but once its run, i.e the password is piped into the command (a python3 command to run an app),...
2
votes
1answer
92 views

How to redirect stdin using mkfifo? [closed]

I searched on askubuntu, stackoverflow and I found this: $ mkfifo fif.pipe $ bash script.sh < fif.pipe $ echo 'say HELLO' > fif.pipe (say command do something in the script) But after $ bash ...
10
votes
2answers
535 views

How to keep colored output using sed

I'm using the sed command and I want to keep colored output from the previous command. The output of ls is colored, but the output of sed is not. I'm using OSX. ls -la | sed -En '/Desktop/q;p'
0
votes
2answers
56 views

Working with grep

I am try to fetch all the links from materialdesignicons.com! I do the following: curl -X GET https://materialdesignicons.com | grep -i "<link href=" | grep -v "<link href=" But it doesn't ...
7
votes
5answers
447 views

Using pv with md5sum

I used md5sum with pv to check 4 GiB of files that are in the same directory: md5sum dir/* | pv -s 4g | sort The command completes successfully in about 28 seconds, but pv's output is all wrong. ...
2
votes
1answer
48 views

how to pipe to a sourced function in bashrc file?

we all know how to pipe to shell scripts and installed programs like grep,awk ... but what about a sourced function in .bashrc or .zshrc ? this is a test function : function fff(){ echo "$@" } ...
0
votes
3answers
257 views

Creating a temporary file from standard input

Say I have a standard input stream of file contents and a command that expects a file name as an argument and I want to run that command on a file made up of the standard input stream’s file contents. ...
1
vote
1answer
46 views

Stream text into a file while compressing it

I have an Arduino Nano connected to a Raspberry Pi which streams text data (the output of a GY-91 sensor) to a listener on the Raspberry Pi. P: 974.044 | T: 22.65 | AG: -0.17 -0.10 1.01 | GD: 1....