Questions tagged [pipe]

A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in <unistd.h>. Shells provide pipe creation between processes using "|".

Filter by
Sorted by
Tagged with
1
vote
1answer
215 views

Bash: child process and pipelining

The purpose of my script is to count how many lines there are in a file. I know I could use wc for example but the purpose of this exercise is to understand processes and pipelines in Linux. The ...
0
votes
1answer
237 views

How to run shell command inside awk and redirect to another shell command?

I want to get particular string from the files downloaded using curl and redirect the downloaded file content to another shell program. Here is the code : curl "http://file-list-url" -s | awk 'BEGIN {...
2
votes
1answer
203 views

Why does echo “a” | wc -m return 2? [duplicate]

Why does echo "a" | wc -m echo 2? Other variations in which I try to reduce the number of spaces yield the same result.
-1
votes
2answers
169 views

Pipe each batch of xargs trough wc -l

So my task is to find the file with the most hardlinks in a directory. So far i have : find . -name "file*" | xargs -I{} -n 1 find . -samefile {} which gives me: ./hardlinkFIle245 ./...
4
votes
2answers
2k views

pipe commands inside find -exec?

Let's suppose I want to find all .txt files and search for some string. I would do: find ./ -type f -name "*.txt" -exec egrep -iH 'something' '{}' \; What if I want to do a more complex filtering, ...
0
votes
1answer
367 views

echo, pipe and command substitution

I want to substitute an echo command followed with | (pipe). For example, $(echo "echo 'hello' | cat") returns 'hello' | cat I expect this to behave like echo 'hello' | cat which returns hello ...
1
vote
2answers
55 views

Two named PIPEs (PIPE_in/PIPE_out) connected with `tail -f` | String sent to PIPE_in doesn't reach PIPE_out

1.Create named PIPEs, pipe_in and pipe_out by running: $ mkfifo pipe_in $ mkfifo pipe_out 2.Connect pipe_in to pipe_out: TERM0: $ tail -f pipe_in > pipe_out 3.Send string hello world! to ...
0
votes
1answer
387 views

Getting the IP address of a given interface [duplicate]

I need the IP (v4) address of a device I specify. The response will be later used in a script. This is what I have: $ IFACE=eth1 $ ifconfig $IFACE |grep "inet " | awk '{print $2}' 10.0.0.33 Is there ...
0
votes
2answers
596 views

How to directly compress CURL output as named archive entry?

I've hundred of text files I need to CURL from remote host. Currently I've a loop which download each file and then compress it as new .zip entry. Is it possible to directly pipe CURL's output to an ...
0
votes
0answers
89 views

C++ script on Spark Cluster throws exit status 132

I'm trying to run a c++ program on spark cluster by using the rdd.pipe() operation but the executors throw: java.lang.IllegalStateException: Subprocess exited with status 132. The spark jar runs ...
1
vote
0answers
100 views

Getting awk value and outputting to a file [duplicate]

I need to get a value from a log file which I am getting using awk at the moment. I can get the number printing on the console but when I try to put that value into a new file nothing happens. Here is ...
0
votes
1answer
956 views

How to write Docker logs to a file in real time (à la `tail -f`)

I docker that output logs stdout stderr, which can be viewed using: docker logs -f $LOGS_CONTAINER_ID I also added 'sed', which puts the container id on each line: docker logs -f $LOGS_CONTAINER_ID ...
0
votes
2answers
247 views

Using bash script to pipe breakpoints to debugger prompt

I am using bash to write a script to make it easier to set breakpoints. I am trying to see if I can use echo and pipe to send set breakpoints commands to the java debugger jdb. The command I have ...
2
votes
2answers
300 views

Redirecting from right to left

From old habits, I'm always redirecting "from left to right", eg. cat file | bar foo | bar I've noticed you can redirect "from right to left". However, is it possible to do this for the second form (...
2
votes
3answers
307 views

Piping Multiple Numbers into Sed

Is it possible to pipe a list of numbers into sed? The pipeline I currently have looks something like: grep -nP 'foo' file_full.txt | sort | awk -F'[:;\t]' '{print $1,$3,$9,$13}' The output of ...
0
votes
1answer
514 views

Socat. Bridge TCP - SERIAL PORT. With log

I'm trying to create a bridge between TCP server and a client connected thorough a serial port using socat. I emulate my TCP server with the following command socat tcp-listen:8888,reuseaddr - I ...
0
votes
1answer
43 views

How to prompt data on piped commands [duplicate]

I'm trying to use this command: mysqldump --opt --hex-blob \ -h source -p xxxx -u user1 \ source_db | \ mysql \ -h target -p yyyy -u user2 \ target_db So, how can I rewrite it to ...
2
votes
1answer
108 views

Reverse the order of file matching as arguments

I have a program that reads files in a certain order. For this example, they all have the same extension, txt {program_name} {input 1...} {output} I happen to name the output file name "a.txt" so it ...
6
votes
3answers
290 views

What characterizes a file in Linux/Unix?

What characterizes a file in Linux/Unix? A file can have many types: regular file, directory, symlink, device file, socket, pipe, fifo, and more that I miss. For example, a symlink: $ sudo file /...
3
votes
2answers
405 views

grep ignoring pipe when aliased to 'grep -R'

I'm trying to pipe the output of a grep search into the input of another grep. Such as: grep search_query * | grep -v but_not_this But the second grep is not using the output of the previous search. ...
0
votes
1answer
175 views

Is it possible to manage total threads for pipes of multi-threaded commands?

I am running several tools that have multi-threading capabilities. For example: tool1 --threads=4 | tool2 --threads=4 | tool3 --threads=4 In an ideal case, I assume this should utilize 12 (4*3) ...
2
votes
1answer
62 views

2 grep calls with pipe doesn't work as expected

I came up with the following command: grep 'rtl' reg_list.txt | grep -E -o -n "id_\w+" reg_list.txt The idea is that first grep lines which contains rtl content, then grep only the words which starts ...
0
votes
1answer
1k views

Why curl > /dev/null and curl -o /dev/null has a large performance difference?

I am using Intel Atom D525, an extremely unpowerful processor for my family router with Debian 9. I tried both curl http://dl.google.com/dl/android/aosp/sailfish-pq2a.190205.003-factory-164a7269.zip &...
1
vote
1answer
164 views

how was the pipe(2) system call working in “traditional” Unix

This is what perldoc -f syscall says: There's a problem with syscall(SYS_pipe()): it returns the file number of the read end of the pipe it creates, but there is no way ...
2
votes
2answers
101 views

Bash: join pipes without deadlock

I want to list a bunch of filenames via find, pipe them through a utility (let's call this util) which outputs a new name for each input name, and then rename each file from the old name to the new. ...
2
votes
2answers
297 views

Bash: How to kill eval if the process that receives its output terminates

I have a little ugly bash script on my Ubuntu machine that contains the lines: search_command="find -L $(printf "%q" "$search_folder") \( ! -regex '.*/\..*/..*' \) -mindepth 1 2> /dev/null" for i ...
26
votes
8answers
3k views

Is piping, shifting, or parameter expansion more efficient?

I'm trying to find the most efficient way to iterate through certain values that are a consistent number of values away from each other in a space separated list of words(I don't want to use an array)....
1
vote
1answer
217 views

Stop netcat as soon as grep matches something

I am facing a problem using netcat in a bash script. I would like to match a specific output after sending a command and continue the script execution as soon as possible (not waiting for a netcat ...
0
votes
0answers
377 views

Piping password multiple times to a command via the command line

I am running Ubuntu 18.04 with bash and I would like to pass a password to a command that runs with sudo, but once its run, i.e the password is piped into the command (a python3 command to run an app),...
1
vote
1answer
317 views

How to redirect stdin using mkfifo? [closed]

I searched on askubuntu, stackoverflow and I found this: $ mkfifo fif.pipe $ bash script.sh < fif.pipe $ echo 'say HELLO' > fif.pipe (say command do something in the script) But after $ bash ...
10
votes
2answers
965 views

How to keep colored output using sed

I'm using the sed command and I want to keep colored output from the previous command. The output of ls is colored, but the output of sed is not. I'm using OSX. ls -la | sed -En '/Desktop/q;p'
0
votes
2answers
80 views

Working with grep

I am try to fetch all the links from materialdesignicons.com! I do the following: curl -X GET https://materialdesignicons.com | grep -i "<link href=" | grep -v "<link href=" But it doesn't ...
6
votes
5answers
871 views

Using pv with md5sum

I used md5sum with pv to check 4 GiB of files that are in the same directory: md5sum dir/* | pv -s 4g | sort The command completes successfully in about 28 seconds, but pv's output is all wrong. ...
3
votes
1answer
199 views

how to pipe to a sourced function in bashrc file?

we all know how to pipe to shell scripts and installed programs like grep,awk ... but what about a sourced function in .bashrc or .zshrc ? this is a test function : function fff(){ echo "$@" } ...
0
votes
3answers
1k views

Creating a temporary file from standard input

Say I have a standard input stream of file contents and a command that expects a file name as an argument and I want to run that command on a file made up of the standard input stream’s file contents. ...
1
vote
1answer
132 views

Stream text into a file while compressing it

I have an Arduino Nano connected to a Raspberry Pi which streams text data (the output of a GY-91 sensor) to a listener on the Raspberry Pi. P: 974.044 | T: 22.65 | AG: -0.17 -0.10 1.01 | GD: 1....
0
votes
1answer
139 views

How does bash coprocess achieve its pipelining?

Note this passage from man bash ( emphasis mine ): Coprocesses A coprocess is a shell command preceded by the coproc reserved word. A coprocess is executed asynchronously in a subshell, as ...
0
votes
0answers
20 views

Using xargs -n and -I together [duplicate]

I have 600 files I wanted to copy to S3. Wanted to do it in batches of 10, so I wrote: $ ls -1 | xargs -ILST -n 10 echo s3 cp LST xx s3 cp LST xx f9b3ac7e-3683-42dd-aed3-41b36bcf822c f9c30a0d-c196-...
2
votes
1answer
395 views

How to create a bidirectional pipe in bash?

I have a program which reads from and writes to file descriptor 3. I want to let it write to fd 3, and be able to write to the other end of the pipe interactively which the program should read over ...
0
votes
1answer
305 views

Pipe file containing list of images to sxiv and reflect changes immediadly

Suppose I have a file images.txt which contains for example image1.jpg image2.jpg image3.jpg Then I pipe the images.txt to sxiv and I want that the thumbnail view reflects the changes immediately. ...
0
votes
2answers
98 views

Save in a file first 7 file in the directory /bin that starts with c

I have write a command in the shell of linux to count the file (in total 8) starting with 'c'in the directory /bin find /bin/[c]* -type f | wc -l What I wish to do now, it's save the name of the ...
0
votes
2answers
359 views

How to let process continue running even after disconnecting ssh?

I have a Linux based router that I’m trying to “control” with a home automation controller. On my automation controller I’m able to utilize a ‘two way strings’ driver to send a string to the Linux ...
-1
votes
1answer
136 views

grep -v is not excluding the strings [duplicate]

im trying to narrow down a flag to catch by removing strings with certain unwanted words so i use cat $(find / -size 33c -group bandit6 -user bandit7) | grep -v 'Permission' but my results end up ...
2
votes
1answer
248 views

pipe of tar and tree commands?

I would like to see the tree of a big compressed file (specifically only the second level of directories) so I used the following command: tar -tf tarfile | tree -L 2 But it outputs the tree of the ...
3
votes
1answer
2k views

how to set parameters when pipe bash script to bash

How to execute bash script with parameters: ./foo.sh a b c When it's compressed (e.g. using xz). xzcat foo.sh | bash <<how_to_supply_here_parameters?>> Specific usecase: I produced ...
3
votes
1answer
149 views

Why is find piping in non directories when the “-type d” test is used?

I have a directory ~/root/ |-- bar |-- eggs |-- foo |-- hello.txt |-- script.sh `-- spam 4 directories, 2 files Issuing find . -type d while in ~/root/ yields . ./spam ./eggs ./bar ./foo However, ...
0
votes
1answer
458 views

How to supply a command to 'stdin' of a running process from a second shell?

If you have a process that is waiting for user-input from the stdin scope, then how can you supply that user-input from a second terminal ? Specifically, if I run the c-program while(1){ fgets(...
0
votes
1answer
1k views

bash and curl commands gets errors : bash: /dev/fd/63: No such file or directory and curl: (23) Failed writing body (0 != 8192)

I am trying to install Litespeed in my Debian instance under Google cloud. I try : sudo bash <( curl https://get.litespeed.sh ) my_serial_number I get : bash: /dev/fd/63: No such file or ...
0
votes
1answer
163 views

Multiple fifos over SSH

Shell script allow multiple file descriptors (up to 9 for POSIX compliance, but even more for bash): $ install -m 755 /dev/stdin /tmp/test <<\EOF #!/bin/sh echo hi1 >&1 echo hi2 >&...
1
vote
1answer
27 views

Using the output of a command to generate an ssh login

I've got a script that I like a lot which generates a two-hopper ssh + port forwarded command. ssh -N -f -o 'ControlMaster Auto' -o 'ControlPath /tmp/gimme_access' -o 'ControlPersist 1m' -L 6473:33....

1
3 4
5
6 7
29