Questions tagged [pipe]

A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in <unistd.h>. Shells provide pipe creation between processes using "|".

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

Why are keystrokes not intercepted after reattaching a "screen -dmS session bash -c 'command'"?

I'm using screen -dmS mysession bash -c 'command > test.txt' to start a screen, launch a command + write its stdout to a file, and detach. Later, after reattaching it with screen -r mysession, I ...
user avatar
  • 2,245
0 votes
1 answer
28 views

Why does this screen session exit when the command is finished? [duplicate]

I know screen -dmS mysession command to start a screen, launch a command and detach. When we need to redirect the stdout to a file, this works: (see How to run a program in a screen, redirect all ...
user avatar
  • 2,245
-1 votes
1 answer
49 views

unzipping piping error on linux system

I am trying to extract windows 10 and install it on linodes platform and my command is as follows: wget -O- --no-check-certificate https://www.dropbox.com/s/e9vh4ahwcogi4xo/Win10_21H2_English_x64.zip?...
user avatar
  • 1
0 votes
0 answers
34 views

Cut off pipe after N bytes

I am piping information to a file using myTool > file.txt 2>&1, but the tool might generate GigaBytes worth of data - I need to cut off after the first N bytes, let say 2MB. It seems pv can ...
user avatar
0 votes
1 answer
23 views

stdin/stderr redirection, piping and color

I've seen a zillion permutations on this question but I think none is an exact duplicate. Can I copy only stderr (or eventually stderr and stdin) to a file, while keeping the colorized output on the ...
user avatar
0 votes
1 answer
25 views

How can I check if the first command on a pipe gets an error?

Let's say I execute the following command on a terminal: this-command-doesnt-exist-and-closes-with-code-127 | jq '' If I execute echo $? I'll get 0 as result because it's checking the exit code from ...
user avatar
  • 319
0 votes
1 answer
26 views

Can I pipe live data from the \watch command of postgresql into another command?

I'd like to pipe live data from psql into another command for purposes of monitoring via the shell. Can this be done? So far I have the following: $ psql bareos -c '\watch (SELECT * FROM log ORDER BY ...
user avatar
  • 4,039
2 votes
1 answer
77 views

GNU Linux "sort -g" can hang indefinitely when run on standard input (on Ubuntu)

The following command (on Ubuntu 20.04 and 22.04): dd if=/dev/urandom count=800000 bs=1 | od -An -t f4 -w4 | sponge | sort -g Can hang indefinitely, with the GNU sort command taking 100% of the CPU. ...
user avatar
  • 121
0 votes
0 answers
18 views

Inquirer program answer questions by piping

Hi I have written this program with inquirer.js it has a list question and after that a text input. I want to answer both questions with piping. If I just keep the list question, I can easily use yes ...
user avatar
0 votes
2 answers
37 views

How do I handle strange path characters when piping find output?

I had a problem where I wanted to find the length of each path from a find command. My first attempt was to run something like this: find . -exec sh -c "echo {} | wc -c" \; I got this idea ...
user avatar
-1 votes
1 answer
27 views

In Bash why echo hello | read myvar && echo $myvar doesn't output? Zsh works fine [duplicate]

I quite don't understand why Bash would not hold the value of myvar in this example. echo hello | read myvar && echo $myvar I would understand if I were running that command as two separate ...
user avatar
0 votes
1 answer
36 views

What is the difference between pipe output to in-place computation? [duplicate]

I am trying to modify my git remote repo address and I got confused over pipe (|). What is the difference between the two and why the first option doesn't work? # Doesn't work git remote get-url ...
user avatar
  • 3
0 votes
0 answers
24 views

pass output as grep pattern [duplicate]

I wish to pass each line of a file as a grep pattern to be searched in data.txt file. Thus, i need cat allsearchstrings.txt | grep <should be lines of allsearchstrings.txt> data.txt cat ...
user avatar
  • 297
0 votes
1 answer
14 views

How to save a website copy into different folders each time using wget?

I have this going in cron: 0 0 1 * * wget --recursive --no-clobber --page-requisites --html-extension --convert-links --no-parent http://example.com -P /foo/bar I'd like save the wget output in a ...
user avatar
  • 135
0 votes
1 answer
32 views

Why pipe doesn't work with upower -e

I've just found this command: upower -e that displays a list of files that can be used with upower -i to display plugged device status. So my first try was using: upower -e | xargs upower -i but it ...
user avatar
  • 8,730
14 votes
2 answers
4k views

Who killed my sort? or How to efficient count distinct values from a csv column

I'm doing some processing trying to get how many different lines in a file containing 160,353,104 lines. Here is my pipeline and stderr output. $ tail -n+2 2022_place_canvas_history.csv | cut -d, -f2 |...
user avatar
  • 193
0 votes
0 answers
24 views

Retain colors from nodejs script after pipeing with sed

I want to include the contents of one JS file inside a nodejs file, so I've done it like so: NodeJS file: let requireJsBundleConfig = REQUIREJS_BUNDLE_CONFIG_CONTENTS ; // ...do stuff Bash ...
user avatar
  • 109
1 vote
2 answers
119 views

Piping into results of another pipe not working as expected

I am a newbie in bash programming and was playing around with some commands trying to learn how passing multiple arguments using pipeline and grouping commands work when I faced this odd behavior ...
user avatar
0 votes
2 answers
20 views

How do I preserve vimcat colors in a pipe?

For example, this will preserve ls's coloring / syntax highlighting: ls -la --color=always | egrep --color=always -i "(foobar|$)" But this will not (so I know it's not a grep issue): vimcat ...
user avatar
1 vote
1 answer
35 views

Redirect Output Back to File [duplicate]

I issued the following command, thinking it would take the contents of foo.txt, process it, and write it back to the original file. Instead when I opened the file, it was empty. cat foo.txt | tr "...
user avatar
  • 115
2 votes
2 answers
80 views

How to grep multiple strings when using with another command?

I am trying to find out how I can use: grep -i With multiple strings, after using grep on another command. For example: last | grep -i abc last | grep -i uyx I wish the combine the above into one ...
user avatar
0 votes
0 answers
24 views

pipestatus requires a value

I tried to add some code to my .zshrc that will create pipenv venv in a project folder. But I'm getting: Error: The argument 'pipestatus <PIPESTATUS> ...' requires a value but none was supplied ...
user avatar
0 votes
1 answer
47 views

Can one pipe compound commands?

My question is: can/how does one pipe compound commands ({ list; })? See Exhibit B. Exhibit A given for comparison's sake. Exhibit A: $ cat << EOF | xargs -n 1 echo foo bar qux EOF ...
user avatar
  • 481
0 votes
1 answer
44 views

What to pipe to strip text?

I download data into a file this way (credit to Valentin Bajrami for the code): curl https://en.wikipedia.org/wiki/Help:Special_page -s | grep -o 'Special:[a-zA-Z0-9]*' | sort -u > ...
user avatar
  • 115
0 votes
1 answer
55 views

How to make a list of values out of a webpage?

In this webpage there is a list of MediaWiki special pages (i.e. pages of the Special: namespace) and I wish to make a simple text list of all these names automatically. I assume that a matching ...
user avatar
  • 115
2 votes
0 answers
38 views

bash: writing to the file descriptor #3 of an unnamed (anonymous) pipe

I need to call a PHP interpreter, which is in a docker container, from my Linux host system. For some reasons I don't want to install PHP on the host system. Now I'm trying to call PHP in this way: ...
user avatar
0 votes
0 answers
10 views

Metadata about process piping into script

If I have a script that accepts piped data, e.g. ls -l, other than stdin is there any way for the script to know about the process that is piping the data to the script? $ ls -l | shelect size, ...
user avatar
  • 14.2k
0 votes
3 answers
251 views

Closing different ends in a pipe

I have wrote the following code for IPC using pipe() : #include <unistd.h> #include <stdio.h> #include <sys/wait.h> int main(void) { char message_buffer[15] = "...
user avatar
  • 87
0 votes
2 answers
88 views

How to search for 3 specific things within a file?

Within the file, security.log I need to find how many IP addresses have the most DNSBL 5 rankings. This is what a line in this file can look like: Oct 24 00:00:28 MAIL-00 postfix/postscreen[10909]: ...
user avatar
0 votes
3 answers
26 views

Sort files by LoC - sort not working when piped from wc -l

I'm trying to sort a directory of files by LoC. But sort appears to do nothing if the lines are piped in: paths=`find ./src/ | egrep "\.(cpp|h)$"` for path in $paths; do wc -l $path | sort -...
user avatar
0 votes
2 answers
60 views

How to pass output of ps x | grep rclone to kill command?

This is the output: $ ps x | grep rclone 7111 ? Sl 0:00 rclone mount xxx 7112 ? Sl 0:00 rclone mount xxx 7113 ? Sl 10:16 rclone mount xxx 9843 pts/1 S+ ...
user avatar
  • 135
2 votes
0 answers
59 views

What is the purpose of a pipe after stdout redirect?

I was reading the documentation for Kaniko and saw a shell redirect-then-pipe construct that I haven't seen before: echo -e 'FROM alpine \nRUN echo "created from standard input"' > ...
user avatar
0 votes
1 answer
120 views

Writing in the file descriptor of the child process (fd/0)

I'm playing with file descriptors in my CentOS 7 sandbox. In doing so, I noticed an interesting situation. Suppose we've a simple PHP script: $step = 4 * 1024; echo "php started\n"; while (!...
user avatar
1 vote
1 answer
50 views

How to compress stdout without block buffering

I tryed this myprogram | gzip > /var/log/mylog.log In my understanding this will write to log file only when a whole gzip block is produced, which I think is 32kB. This introduces considerable lag ...
user avatar
  • 123
1 vote
0 answers
25 views

Is there any handy way to follow multiple piped commands' status output

Let's say I have serverA with a block device, the contents of which I want to move to serverB's block device via the computer I am working on as a relay. This is very handy with pipes, buffers, dd and ...
user avatar
0 votes
2 answers
58 views

Unable to match a specific regex with bash

I have a txt file with content as such: Adedunmola Okikiola Adewole 512.035 215−39 ^M Ademir Cleto de Oliveira 055.735 445−13 ^M Adilson ...
user avatar
  • 143
0 votes
1 answer
40 views

How to write a script such that I can pipe it to find?

I have a script that expects files as arguments, and for each executes a set of instructions. How should I write it to be able to pipe it to find? In the example below, I haven't used find as such. I ...
user avatar
  • 481
0 votes
1 answer
63 views

execlp "sort" with input from pipe stucks, why?

sort is waiting, but what? I tried execlp("head", "head", "-n", "3", NULL); instead sort and it works fine. #include <stdlib.h> #include <sys/wait.h&...
user avatar
  • 3
2 votes
1 answer
28 views

Send stdout to multiple downstream processes in a chain

I have a series of commands I run like this: cmd1 < input > foo cmd2 < foo > bar cmd3 foo bar > output Is there a way to do this without the intermediate files foo and bar? I'd also ...
user avatar
  • 123
0 votes
2 answers
99 views

Ansible -m shell and find command

Does anybody know how to solve this problem? ansible -m shell -i test.txt -a "sudo find / -regex '.*\(tar\|zip\)$' -type f 2>/dev/null | while read i; do echo $i; done" The problem is ...
user avatar
  • 25
2 votes
1 answer
42 views

Close all readers of a FIFO once all its input is read?

How can I close all readers of a FIFO once all its input is read? Seems I can only get one of them to close, which keeps my program from finishing. Here's a sample program that works (to test put it ...
user avatar
  • 155
3 votes
1 answer
51 views

aborting previous steps in curl, xargs pipe when head finishes

I'm trying to download a bunch of web pages, and once I've downloaded N lines of html, I want the whole thing to stop. But instead, the previous steps in the pipe just keep going. An example to see ...
user avatar
  • 155
1 vote
0 answers
61 views

Capture arguments after PIPE in a bash script as string

I am writing a simple script (well, I thought it would be simple) to gather and log a 'reason' for using sudo. That part is working pretty straightforward based on another article I found (https://...
user avatar
0 votes
0 answers
62 views

grep with pipe and variables NOT working on remote server through ssh (working locally)

Below grep command is not giving expected result (matching line number) if I run it through ssh from a remote machine (passwordless authentication enabled). Variables have been defined correctly as ...
user avatar
4 votes
2 answers
250 views

ssh reads stdin without being asked to

These do not do the same: $ seq 1000000 | (ssh localhost sleep 1; wc -l) 675173 $ seq 1000000 | (ssh localhost sleep 1 </dev/null; wc -l) 1000000 What is the rationale for ssh reading stdin?
user avatar
  • 29.6k
0 votes
1 answer
424 views

jq - pipe multiple files from "find" to jq

I try to get all versions of a specific node-package within a project nodes_module folder. There are multiple package.json files and I wish to "grep" the ones under "dependencies": ...
user avatar
  • 3
0 votes
1 answer
111 views

get count of lines while in pipe

i have a bash script that looks like this: some_process | sort -u | other_process >> some_file I'd like to get the line count to while i'm streaming the data, after the data is sorted but ...
user avatar
  • 195
0 votes
2 answers
65 views

Incremental reading from pipe

Trying to figure out how to have a shell script only take fix-length blocks out of stdin. I would have thought something like this would have worked, but it does not: #!/bin/bash value=0 while [ &...
user avatar
  • 196
1 vote
0 answers
45 views

How to extract a value from a piped output to a variable repeatedly while the output is still being written

I want to update the download status at every 5 second of file to my telegram bot. Also here I'm using bash. aria2c $url --summary-interval=5 2>&1 | tee output.log | grep -oP "(\d+(\.\d+)?(...
user avatar
  • 11
0 votes
1 answer
57 views

File descriptor 10 when running a script (Bash/dash, etc.)

When the script runs, a file descriptor 10 appears with the contents of the executable script. For example, the script: #!/bin/sh sleep 600 When viewing the fd of a running script through processes, ...
user avatar
  • 127

1
2 3 4 5
35