Questions tagged [pipe]

A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in <unistd.h>. Shells provide pipe creation between processes using "|".

Filter by
Sorted by
Tagged with
2
votes
3answers
76 views

Splitting a file into two variables for piping to one command

I have a JSON object. it has two fields; { href: string, filename: string }. I want to download a file from href, and set its name to $FILENAME.pdf. This happens in a loop How can I do that? ...
1
vote
1answer
247 views

How can I get feh to set wallpaper to each monitor according to its size?

I have a directory filled with png files $ tree ~/wallpaper/ ~/wallpaper/ ├── foo--1366x768.png ├── foo--1920x1080.png ├── foo--2048x1080.png ├── foo--3440x1440.png └── foo--3840x2160.png 0 ...
1
vote
2answers
262 views

Entr: trying to trigger function while monitoring file change

I'm trying to monitor some files with entr. My script based on their examples: do_it(){ echo Eita!; } while true; do ls folder/* more-folder/* | entr -pd do_it; done >> entr: exec do_it: No ...
0
votes
4answers
178 views

How to pipe fd 3 to consumer instead of fd1 / fd2

I was hoping to do something like this: echo 'foo' >&3 3| cat Basically, I want to write 'foo' to 3, and then only pipe the data in 3 to cat. But the above doesn't work, I get: bash: 3: ...
1
vote
2answers
57 views

output mixed alphanumeric input with grep,pipe and cat

I have a mixed wordlist as an input: azert12345 a1z2e3r4t5 a1z2e3r455 The command line I have tried to execute: cat file.txt | grep -E "[[:digit:]]{5}" --color What do I want to accomplish: Print ...
0
votes
0answers
46 views

output only the mixed alphanumeric input with grep,pipe and cat [duplicate]

I have a mixed wordlist as an input azert12345 a1z2e3r4t5 a1z2e3r455 what i need to do is to print only the mixed alphanumeric words mad of 5 digits and 5 alphabets like (a1z2e3r4t5, azert12345) ...
-2
votes
1answer
65 views

tar flexibility for -f

In the following command for example, tar -xzv works but tar -xzvf fails: wget ${target_url} -O - | tar -xzv --strip-components 1 -C ${web_application_root}/${domain} I understand I download a data ...
1
vote
1answer
38 views

Pipe inputs from more than 1 process

I am trying to automate some make commands. Along with the target we have a variable that is passed to make (corresponding to set of a #ifdef's in code). The make command looks like this: make <...
-2
votes
3answers
444 views

Piping of stream from wget to tar, and extracting it to a specific location failed

I try to pipe a stream from wget to tar and extracting it to a specific location. The file is downloaded by wget but not extracted as desired with tar: war="/var/www/html" domain="example.com" ...
0
votes
1answer
49 views

How to specify output file name using piping?

I am using find to return the list of files from a directory and its subdirectories: find $directory -type f -name "*.epub" Then I want to use a command that requires the input and output filenames ...
0
votes
0answers
25 views

How can I redirect output from the end of a pipe to the input of the beginning of the pipe [duplicate]

I'm trying to learn the intricacies of piping and redirection, and there's one thing I can't seem to manage to do. I'd like to try to get a loop going such that AA stdout -> BB stdin BB stdout -> ...
1
vote
1answer
77 views

Piped command hangs on killed sleeping child

I use a 'watchdog' function to perform a failsafe wakeup if a certain command hangs. This is implemented by running a background task that sleeps for 30s and then executes the wakeup. If the parent ...
1
vote
0answers
30 views

How can I create a function that memoizes other commands?

I have created the following function to memoize any command: # zsh (( $+functions[ec] )) || { ec() print -r -- "$@" ecerr() { ec "$@" 1>&2 } ecdbg() ecerr "$@" gq() { ec "${(q+...
7
votes
3answers
558 views

Find a text string in a file and output only the rest of the text that follows it?

I am finding a lot of posts on this forum that have to do with finding various values in a text file and outputting text surrounding it. However, I don't seem to find any "stream oriented". I want ...
0
votes
0answers
32 views

Can't pipe stdout of a python script that runs in background to file

I have a python script myscript.py on a RaspberryPi with Raspbian that starts on boot via rc.local file. Since it has infinite loop in it I need to run it in background. So far so good but I also want ...
0
votes
0answers
36 views

producer and consumer write/read at same rate

I would like create a pipeline where the producer writes at the same rate the consumer reads..for example: tail -f -n +1 | my_proc the tail command will write all requested lines to my_proc and ...
3
votes
3answers
232 views

How to save command as well as output of that command to a file [duplicate]

Hi I want to save both output of a command and the command that has been run to create the output into the same single file. I understand either of *some_command* > file.txt *some_command* | ...
1
vote
0answers
112 views

When piping to a command via tee, output of first command is lost

The following command displays output of only grep command. df|tee >(head -1) |grep -i '/dev' whereas the following command gives output of both grep and head command df|tee >(head -1) >(...
0
votes
2answers
237 views

redirecting `ls` output to another command with xargs

In the directory I am working on I have two files with the extension .sam: PD180425_aligned_minimap.sam PD180793_aligned_minimap.sam For each of these two files, I need to apply a command that looks ...
1
vote
2answers
372 views

Can more than one process in a pipeline read from the stdin of the terminal?

Say I run a pipeline from a terminal: $ a | b is there a way for me to send a signal from the terminal to both processes, or can stdin from the terminal only be read from one of the two processes? ...
3
votes
1answer
38 views

how to keep consumer alive even if producer exits

Say we have a simple pipeline: a | b say, a exits - is there a surefire way to keep b alive for an arbitrary amount of time (to complete a task etc).
0
votes
1answer
41 views

At what point can the consumer in a pipeline slow down the producer?

Say we have a | b, under what circumstances could b slow down a? If b was a CPU hog, that would do it...what if b was consuming stdin really slowly? are there other factors?
0
votes
2answers
54 views

Piped greps for looking inside files

(self-migrated from ask-ubuntu because it's linux-related, not ubuntu, and my os isn't ubuntu) I'm trying to make a grep that looks like this: grep -r 2019 | grep -riv FAILED | grep -rl DSL I want ...
0
votes
1answer
41 views

Pass a group of bash commands to bash function

I want to pass the following group of bash commands { echo Apple; echo Banana; } as an argument to a bash function defined in .bashrc as follows: BashFunction(){ "$@" | SomeOtherFunction } ...
1
vote
0answers
33 views

How to determine if a process is part of a pipeline (producer in pipeline)? [duplicate]

So if I am in my terminal and run: $ my_proc | grep foo in my_prof I know it's part of a pipeline because stdout is not attached to the terminal but to grep. On the other hand, if I just run this: $...
2
votes
1answer
339 views

Piping grep output into grep inside a bash

I've got a text file, and I want to search for lines that have search_string but do not have exclude_string. In terminal, the following gives me the output I want: grep "search_string" | grep -v "\(...
0
votes
1answer
101 views

Pass piped command line to xargs

I have this command line: curl https://token-api/42 | awk '{ print $0 }' which produces this output: {"id":"PX-12345","expiresAtUtc":"2020-07-05T06:55:52","createdUtc":"2019-07-15T06:55:52"} The ...
5
votes
1answer
735 views

Writing to disk and compress with xz at the same time

I have a program that writes traces on disk and the size becomes very large. Normally, I use the following commands. ./run output.txt xz output.txt Can I pipe xz at the same time as output.txt is ...
0
votes
0answers
263 views

RecordIO/ protobuf format file

i am using the aws sagemaker and it requires to convert my data to RecordIO/protobuf format. So i did but not its giving me errors on the test file that it does not start with a valid magic number. so ...
1
vote
1answer
39 views

Copy file with changed content

I'm writing bash script where I need to store some files as a .bak files and change it content at the beginning(using sed for that). I'm looking for better way to write it down for my bash script. cp ...
0
votes
2answers
49 views

sed deleting everything before ip address of host

I'm looking to delete every line before the IP address of the local host. I'm grepping the host's IP address successfully via: grep `hostname` /etc/hosts | awk '{print $1}' Now I'm looking how to ...
0
votes
2answers
135 views

Can I pipe stdout to the middle of a command with emacs?

The command emacs -nw foo -f org-mode opens the file foo in emacs in org-mode. Now, suppose that the filename foo is the output of a script. How could I pipe to my command? So, I'm trying to get ...
1
vote
1answer
553 views

convert an excel file to pipe delimiter file in Linux using script

Need to convert an excel file to pipe delimiter file in Linux using shell script. pls help I'm new to shell script. I saw a post to convert comma to pipe using the below code... awk '{for(i=1; i<...
0
votes
0answers
39 views

How to create program that automatically removes old lines

Say I have a simple pipeline: receive-data | write-to-file --max-lines 500 -f <file> how can I implement a program write-to-file where the max number of lines is 500 or some number? (It can ...
0
votes
0answers
12 views

Close pipeline gracefully from the consumer side

I have a simple pipeline tail -f <file> | node myapp.js in my node.js code, I have some hack that less uses to listen to stdin from the terminal: const fd = fs.openSync('/dev/tty', 'r+'); ...
0
votes
0answers
56 views

How to portably test for a SIGPIPE failure

In a bash script, I have a pipeline of this form /bin/find ... | grep --quiet --max-count=1 . Basically, I am just testing, yes/no, whether /bin/find found anything. Therefore, if /bin/find fails ...
0
votes
1answer
39 views

Piping tar archive via mv on macOS

I ran tar -cjvf ~/Dict_$(date +%Y-%m_%d-%H_%M_%S).tar.bz2 ~/Dictionaries | xargs mv ~/Documents but it leaves the resulting file in ~/ and does not move it. How do I move the resulting file (after ...
1
vote
1answer
207 views

how to pipe output of ps2pdf to zathura

I want to view manpages as pdf. I tried the following: man -t awk | ps2pdf - - | zathura (combination of this and the last example in the manpage of ps2pdf) However, zathura just shows me a black ...
0
votes
0answers
32 views

How does `less` listen to stdin commands [duplicate]

Say I run this: node file.js | less the less process can listen for stdin / keystrokes, for example if I hit q for quit or / to search for keywords. For example if I search for /p I see: So how ...
1
vote
1answer
50 views

Can I safely git/commit/push or rsync a file being currently written by tee without breaking any pipe?

I have a pipe working for several days and writing some scientific results at a very low rate (a block of about 20 lines once an hour); most of the time, nothing is written. The file is written by tee....
5
votes
2answers
134 views

Linux: write tail -f output, plus comments, to a separate text file

The command tail -f command is great for tracing, and in terminal it's very helpful to be able to, for example, hit enter, and type a comment like -- after xyz change... -- For record purposes, I ...
0
votes
0answers
18 views

Piping to subl works only one time

I tried to run ls | subl <(cat) and it opened Sublime Text with ls output. But if I run ps | subl <(cat) I don't get it if editor is already opened. I would expect it opens new tab for new ...
0
votes
0answers
61 views

Understanding anonymous pipe ID

I'm currently debugging a system which has a resource leak. It seems that we have too many pipse open. When I go the my /proc/PROC_ID/fd I see the list of pipes l-wx------ 1 root root 64 Jun 30 12:32 ...
1
vote
1answer
55 views

How to pipe app1 to app2, but show both output in terminal? [duplicate]

I would like to know how can I redirect the stdout of an app to an input of an another app but still see the app1 output. The source app is written in c, the second app is in perl. The purpose of ...
0
votes
1answer
101 views

Using `read` in bash script from pipeline

Bash can run scripts from pipeline. A minimal example would be: > echo "echo hello world" | bash hello world This is handy if you wants to run a script downloaded from curl or wget. But ...
0
votes
0answers
43 views

Separate stdout of an application

I have an application that takes a few command line parameters, then prints some text to the terminal, then starts writing data to a file. It has a parameter I can use to define the file name to write ...
0
votes
0answers
73 views

Redirect output of a long running process to a new log file each day

We have a long running process1 whose output is captured into a log file based on when the process is started: export LOG_FILE=${LOGS_DIR}/$LOGNAME/${LOGNAME}-`date +%Y%m%d-%H%M%S`.txt ... ${...
12
votes
2answers
528 views

Parameterize chained calls to a utility program in Bash

I have a black box UNIX program used in a Bash shell that reads columns of data from stdin, processes them (applying a smoothing effect) then outputs to stdout. I use it by UNIX pipes, like ...
0
votes
1answer
83 views

get all git remote.origin.url from exisiting clones in sub directory

i like to write a oneliner which gets me all remote url from my cloned repositories underneath my git_repo folder (personal git root folder) and to pipe into a file my idea was something like cat */....
1
vote
1answer
52 views

How “some_command | sudo tee” really works?

I guess I know when there is a pipe (|) the left-side command is executed first and the result is passed to the right-side command of pipe but... apt update | sudo tee ... how can work that way? ...

1 2
3
4 5
29