Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [pipe]

A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in <unistd.h>. Shells provide pipe creation between processes using "|".

0
votes
1answer
16 views

How to prompt data on piped commands

I'm trying to use this command: mysqldump --opt --hex-blob \ -h source -p xxxx -u user1 \ source_db | \ mysql \ -h target -p yyyy -u user2 \ target_db So, how can I rewrite it to ...
2
votes
1answer
42 views

Reverse the order of file matching as arguments

I have a program that reads files in a certain order. For this example, they all have the same extension, txt {program_name} {input 1...} {output} I happen to name the output file name "a.txt" so it ...
6
votes
3answers
162 views

What characterizes a file in Linux/Unix?

What characterizes a file in Linux/Unix? A file can have many types: regular file, directory, symlink, device file, socket, pipe, fifo, and more that I miss. For example, a symlink: $ sudo file /...
2
votes
2answers
50 views

Grep Ignoring Pipe?

I'm trying to pipe the output of a grep search into the input of another grep. Such as: grep search_query * | grep -v but_not_this But the second grep is not using the output of the previous search. ...
0
votes
1answer
18 views

Is it possible to manage total threads for pipes of multi-threaded commands?

I am running several tools that have multi-threading capabilities. For example: tool1 --threads=4 | tool2 --threads=4 | tool3 --threads=4 In an ideal case, I assume this should utilize 12 (4*3) ...
2
votes
1answer
41 views

2 grep calls with pipe doesn't work as expected

I came up with the following command: grep 'rtl' reg_list.txt | grep -E -o -n "id_\w+" reg_list.txt The idea is that first grep lines which contains rtl content, then grep only the words which starts ...
0
votes
1answer
47 views

Why curl > /dev/null and curl -o /dev/null has a large performance difference?

I am using Intel Atom D525, an extremely unpowerful processor for my family router with Debian 9. I tried both curl http://dl.google.com/dl/android/aosp/sailfish-pq2a.190205.003-factory-164a7269.zip &...
1
vote
1answer
73 views

how was the pipe(2) system call working in “traditional” Unix

This is what perldoc -f syscall says: There's a problem with syscall(SYS_pipe()): it returns the file number of the read end of the pipe it creates, but there is no way ...
2
votes
2answers
53 views

Bash: join pipes without deadlock

I want to list a bunch of filenames via find, pipe them through a utility (let's call this util) which outputs a new name for each input name, and then rename each file from the old name to the new. ...
2
votes
2answers
100 views

Bash: How to kill eval if the process that receives its output terminates

I have a little ugly bash script on my Ubuntu machine that contains the lines: search_command="find -L $(printf "%q" "$search_folder") \( ! -regex '.*/\..*/..*' \) -mindepth 1 2> /dev/null" for i ...
23
votes
8answers
2k views

Is piping, shifting, or parameter expansion more efficient?

I'm trying to find the most efficient way to iterate through certain values that are a consistent number of values away from each other in a space separated list of words(I don't want to use an array)....
1
vote
1answer
43 views

Stop netcat as soon as grep matches something

I am facing a problem using netcat in a bash script. I would like to match a specific output after sending a command and continue the script execution as soon as possible (not waiting for a netcat ...
0
votes
0answers
37 views

Piping password multiple times to a command via the command line

I am running Ubuntu 18.04 with bash and I would like to pass a password to a command that runs with sudo, but once its run, i.e the password is piped into the command (a python3 command to run an app),...
2
votes
1answer
51 views

How to redirect stdin using mkfifo? [on hold]

I searched on askubuntu, stackoverflow and I found this: $ mkfifo fif.pipe $ bash script.sh < fif.pipe $ echo 'say HELLO' > fif.pipe (say command do something in the script) But after $ bash ...
10
votes
2answers
487 views

How to keep colored output using sed

I'm using the sed command and I want to keep colored output from the previous command. The output of ls is colored, but the output of sed is not. I'm using OSX. ls -la | sed -En '/Desktop/q;p'
0
votes
2answers
48 views

Working with grep

I am try to fetch all the links from materialdesignicons.com! I do the following: curl -X GET https://materialdesignicons.com | grep -i "<link href=" | grep -v "<link href=" But it doesn't ...
7
votes
5answers
402 views

Using pv with md5sum

I used md5sum with pv to check 4 GiB of files that are in the same directory: md5sum dir/* | pv -s 4g | sort The command completes successfully in about 28 seconds, but pv's output is all wrong. ...
2
votes
1answer
27 views

how to pipe to a sourced function in bashrc file?

we all know how to pipe to shell scripts and installed programs like grep,awk ... but what about a sourced function in .bashrc or .zshrc ? this is a test function : function fff(){ echo "$@" } ...
0
votes
3answers
149 views

Creating a temporary file from standard input

Say I have a standard input stream of file contents and a command that expects a file name as an argument and I want to run that command on a file made up of the standard input stream’s file contents. ...
1
vote
1answer
37 views

Stream text into a file while compressing it

I have an Arduino Nano connected to a Raspberry Pi which streams text data (the output of a GY-91 sensor) to a listener on the Raspberry Pi. P: 974.044 | T: 22.65 | AG: -0.17 -0.10 1.01 | GD: 1....
0
votes
1answer
40 views

How does bash coprocess achieve its pipelining?

Note this passage from man bash ( emphasis mine ): Coprocesses A coprocess is a shell command preceded by the coproc reserved word. A coprocess is executed asynchronously in a subshell, as ...
0
votes
0answers
18 views

Using xargs -n and -I together [duplicate]

I have 600 files I wanted to copy to S3. Wanted to do it in batches of 10, so I wrote: $ ls -1 | xargs -ILST -n 10 echo s3 cp LST xx s3 cp LST xx f9b3ac7e-3683-42dd-aed3-41b36bcf822c f9c30a0d-c196-...
2
votes
1answer
61 views

How to create a bidirectional pipe in bash?

I have a program which reads from and writes to file descriptor 3. I want to let it write to fd 3, and be able to write to the other end of the pipe interactively which the program should read over ...
0
votes
1answer
50 views

Pipe file containing list of images to sxiv and reflect changes immediadly

Suppose I have a file images.txt which contains for example image1.jpg image2.jpg image3.jpg Then I pipe the images.txt to sxiv and I want that the thumbnail view reflects the changes immediately. ...
0
votes
2answers
23 views

Save in a file first 7 file in the directory /bin that starts with c

I have write a command in the shell of linux to count the file (in total 8) starting with 'c'in the directory /bin find /bin/[c]* -type f | wc -l What I wish to do now, it's save the name of the ...
0
votes
2answers
73 views

How to let process continue running even after disconnecting ssh?

I have a Linux based router that I’m trying to “control” with a home automation controller. On my automation controller I’m able to utilize a ‘two way strings’ driver to send a string to the Linux ...
-1
votes
1answer
42 views

grep -v is not excluding the strings [duplicate]

im trying to narrow down a flag to catch by removing strings with certain unwanted words so i use cat $(find / -size 33c -group bandit6 -user bandit7) | grep -v 'Permission' but my results end up ...
2
votes
1answer
50 views

pipe of tar and tree commands?

I would like to see the tree of a big compressed file (specifically only the second level of directories) so I used the following command: tar -tf tarfile | tree -L 2 But it outputs the tree of the ...
3
votes
1answer
235 views

how to set parameters when pipe bash script to bash

How to execute bash script with parameters: ./foo.sh a b c When it's compressed (e.g. using xz). xzcat foo.sh | bash <<how_to_supply_here_parameters?>> Specific usecase: I produced ...
3
votes
1answer
139 views

Why is find piping in non directories when the “-type d” test is used?

I have a directory ~/root/ |-- bar |-- eggs |-- foo |-- hello.txt |-- script.sh `-- spam 4 directories, 2 files Issuing find . -type d while in ~/root/ yields . ./spam ./eggs ./bar ./foo However, ...
0
votes
0answers
66 views

How to supply a command to 'stdin' of a running process from a second shell?

If you have a process that is waiting for user-input from the stdin scope, then how can you supply that user-input from a second terminal ? Specifically, if I run the c-program while(1){ fgets(...
0
votes
1answer
114 views

bash and curl commands gets errors : bash: /dev/fd/63: No such file or directory and curl: (23) Failed writing body (0 != 8192)

I am trying to install Litespeed in my Debian instance under Google cloud. I try : sudo bash <( curl https://get.litespeed.sh ) my_serial_number I get : bash: /dev/fd/63: No such file or ...
0
votes
0answers
59 views

Multiple fifos over SSH

Shell script allow multiple file descriptors (up to 9 for POSIX compliance, but even more for bash): $ install -m 755 /dev/stdin /tmp/test <<\EOF #!/bin/sh echo hi1 >&1 echo hi2 >&...
1
vote
1answer
23 views

Using the output of a command to generate an ssh login

I've got a script that I like a lot which generates a two-hopper ssh + port forwarded command. ssh -N -f -o 'ControlMaster Auto' -o 'ControlPath /tmp/gimme_access' -o 'ControlPersist 1m' -L 6473:33....
0
votes
1answer
63 views

Awk doesn't exit despite “exit 0” command

I have the following command: sudo docker logs -f <id> | awk '/Listening on / {print $3; exit 0;}' From what I understand, awk should find the first line that matches the pattern, print the ...
1
vote
1answer
133 views

Virtual serial port between docker containers

I can create two linked virtual serial ports on a Linux system with socat, and pretend one end is a serial device, and the other one is some code that uses the device. The socat command would look ...
1
vote
1answer
51 views

pipe into a curses program

The library ncurses provides an interactive user interface. However, I was wondering what would happen if you pipe something into the program. So i tried like this: # echo foo | my_curses_prog It ...
0
votes
1answer
18 views

using sudo with piped command on Ubuntu server

If I need to use the following backup command on an Ubuntu server with sudo, how do I do so? Do I need sudo at two places? tar cf - directory | 7za a -si directory.tar.7z
1
vote
0answers
33 views

Killing a process when some other process is finished

Given a | b, i'd like to kill b when a is finished. b is an interactive process, which doesn't terminates when a is finished (fzf in my case), and the whole a | b is executed in a $() subshell. So ...
0
votes
0answers
31 views

while read loop seems to terminate after the first line, when using two pipes?

#!/usr/bin/env bash set -euo pipefail while read -r line do echo "line via echo:" echo "$line" echo "line via sed:" sed 's/w/f/g' | sed 's/f/zzzzz/g' <<< "$line" done < /dev/...
0
votes
0answers
22 views

Buffering issue when looking at netcat output

I have problem I am trying to understand: I am sending some UDP data to local host. It is a simple UDP packet with a counter. It counts up to 16 (0xF) and stops. When I grep the received data on the ...
0
votes
2answers
57 views

STDOUT + STDERR output … is there any difference between considering the output to be an empty string vs NULL

I'm writing some application code that is used to execute Linux shell commands, and it then logs the command details into an SQL database. This includes the output of STDOUT + STDERR (separately). ...
0
votes
3answers
36 views

Exit pipeline early on command failure

I looked at Exiting a pipeline if a former command fails but I'm looking for a more general solution to this problem: chlist -q -v -m -r $sourcelist | \ grep '^USER:' | \ chlist -q -v -f - $superlist ...
1
vote
1answer
40 views

Bit banging with dd?

I need to change one byte at a well-known position of a binary file to a specific value, and am looking for a way to achieve that using command line tools. Constraints: I need to change one single ...
0
votes
2answers
116 views

Why doesn't grep remove lines of terminal output from find command by default? [duplicate]

I am constantly frustrated by this simple command: find / | fgrep somestuff.ext When I don't use sudo, I get line after line of permission denied - which is fair enough, but why isn't this output ...
0
votes
2answers
38 views

How do I diff the output of these two commands? [duplicate]

I have an alias which I use to strip out comments that are piped to it: alias decomment='egrep -v "(^#.*|^$)"' I'm currently having some difficulty with a squid proxy setup, so I want to compare ...
1
vote
1answer
21 views

A command wants file paths. How can I give it stdin for the “infile” and stdout for the “outfile”?

Consider a command like foo -in /path/to/infile -out /path/to/outfile, that you'd like to pass strings to rather than using temp files. It may be called a lot causing much disk access, or is related ...
-3
votes
2answers
379 views

Calling sh with script on stdin and passing command line arguments

This works of course. I can run ./test1.sh cats and the argument cats is passed correctly to ls. $ cat test1.sh #!/bin/sh ls -l $1 Can you tell me how do I make this second example — the same ...
0
votes
2answers
93 views

sed not working correctly when piped

For some reason, I will not have live output when piping the command output of e.g. SSH to sed: ssh someuser@somehost 2>&1 | sed -e "s/\[32//g" | tee logging I suspected that the output dit ...
0
votes
2answers
77 views

wget and scp as a pipeline

Can we use wget and scp as a pipeline. I want to wget a file from a server and copy to another server. I used below command but it is not working. wget "$Select_Release_Version_ARTIFACT_URL" | sudo ...