A Unix pipe connects file descriptors of two processes. A pipe is created with the POSIX pipe() function declared in . Shells provide pipe creation between processes using "|".

learn more… | top users | synonyms

27
votes
7answers
32k views

how do I trim leading and trailing whitespace from each line of some output?

I would like to remove all leading and trailing spaces and tabs from each line in an output. Is there a simple tool like trim I could pipe my output into?
0
votes
2answers
54 views

How to grep stderr? [duplicate]

How do I grep this command that includes a wildcard? How do I grep stderr? The grep does not seem to do anything: $ less /Users/raine/.nvm/versions/node/v5.5.0/lib/node_modules/* | grep "Not a file"...
1
vote
1answer
12 views

Ldap error doesn't redirect to file

I'm running a loop querying ldap server for a single dn every time, just to see if it's running okay. I want to output both the successful and unsuccessful responses to a file and while the former ...
1
vote
0answers
56 views

How to send sudo password over pipe? [duplicate]

I need to replace existing sshd config on the remote PC and restart sshd, so I try to use this: cat /sshd_cfg | ssh user@192.168.0.47 "cat > /etc/ssh/sshd_config && service sshd restart" ...
2
votes
2answers
52 views

Pass input file through pipe as argument?

I have two issues. I am trying to pass files through a pipe as an argument, and I am trying to use that file as a variable inside the sh function. Here is my command: find . -name 'segment*' | ...
4
votes
3answers
167 views

Why does `ps -e | less` show 'less' among processes? [duplicate]

I run ps -ejH | less. The output includes ps and less as well. What is the reason? I thought it would work as follows: First, ps will run and it will list all processes existing at that moment. ...
3
votes
1answer
59 views

set environment variables from stdout

How can you do the equivalent of piping stdout to 'export'? For background, I have a non-shell script that generates environment variables like so: DATABASE_URL='someurl' MAIL_KEY='key' REDIS_URL='...
10
votes
2answers
4k views

echo vs <<<, or Useless Use of echo in Bash Award?

By now the Useless Use of cat Award is very well known, and there's also a mention of a Useless Use of echo (not relevant for this question). I'm wondering if there should be a "Useless Use of echo in ...
0
votes
1answer
39 views

What's the proper way to read stdin inside parenthesis assignment of 2nd pipe

1st style, group 2 pipe to assign, this works: $ myresult1=$(curl -L 'https://archive.org/wayback/available?url=stackoverflow.com' 2>/dev/null | jq '.archived_snapshots.closest.url'); echo "$...
5
votes
0answers
36 views

Piping the output of a bash script after redirecting stderr

because of the way my hoster does things I have to use a system for services. This service has a run file which it executes and pipes into a second run file. But I use a (given) python package that ...
1
vote
0answers
16 views

strace for troubleshooting inter process communication

I have output captured by following command: strace -f -e trace=process,socketpair,open,close,dup,dup2,read,write -o rsync.log rsync -avcz --progress src/ dst/ it is a bit long so I've uploaded it ...
5
votes
2answers
17 views

Can you scp, sftp, or rsync, a pipe?

I want to tar and send a ~700GiB directory to a remote drive I don't control. I don't have the HDD space locally to create the tarball and then copy that over. The remote is also protected by rssh - ...
1
vote
3answers
2k views

Ls with spaces + variables

I want to do something like this, but it doesn't save the variable after the piping ends: fs=( ) echo ${fs[@]} ls -A1 | while read f do echo ${fs[@]} fs+=( "$f" ) echo ${fs[@]} done echo "...
0
votes
1answer
17 views

SQLite command line: How do you pipe output to a UNIX utility?

The SQLite documentation for its command line client indicates that its possible to filter output of SQLite queries through UNIX utilities: The default output mode is "list". [...] List mode ...
2
votes
2answers
876 views

Mplayer controls when using stdin playlists

I ran across this post: How can I retain the console input in mplayer when reading from stdin? But the answer there doesn't work for me. I'm running the following command: ls -l | grep -e mp3 -e ...
0
votes
0answers
12 views

F_GETPIPE_SZ returns -1 in C [migrated]

Why would F_GETPIPE_SZ return -1? It sounds like an error, but I can't find any mention of what error it is, or, more importantly, what I'm supposed to do to not get the error. I'm running Raspbian ...
0
votes
1answer
26 views

Error while replacing date in a file using tr command

I want to replace date in a file with my date. For this, I am giving below command - tr '2016-05-25' '2016-06-17' < APCSE060.20160525 > TESTFILE alternatively cat APCSE060.20160525 | tr '...
15
votes
1answer
754 views

How to terminate a TCP connection established by bash itself?

I have used exec 3<>/dev/tcp/192.168.0.101/6435 to establish a TCP connection with 192.168.0.101:6435. And I have received as well as sent a few messages with the pipe command. Now, I want to ...
1
vote
1answer
34 views

netcat doesn't print response

I'm trying to send commands to a tcp port using netcat and pipe response when I run netcat and type my command it prints response correctly but when I pass command from a pipe it sends the command ...
1
vote
2answers
473 views

using “ifne” in Bash pipe

In my script, I am using the ifne utility from the moreutils package. The line can be simplified to the following: printf "asdf\n" | ifne cat - && echo "stream not empty" ifne only executes,...
1
vote
0answers
25 views

Use a result twice in a pipeline [duplicate]

I have two programs progA and progB ./progA outputs a list like this : a1 a2 a3 a4 and ./progB take the result of progA as an argument and also outputs a list : b1 b2 b3 b4 I can pipe them like ...
4
votes
2answers
36 views

How would I get my terminal to regurgitate the previous output text from past commands? Is this even possible?

I just ran a job that takes several hours, and I forgot to pipe that text into a text file. pseudocode: echo [previous text output] > OutputHistory.txt Additionally, I can't just "copy and ...
23
votes
14answers
31k views

What's a good example of piping commands together?

If you were helping someone to learn the concept of pipes on the command line what example would you use? The example that actually came up was as follows: cat whatever.txt | less I feel like that'...
1
vote
1answer
36 views

Pipe blocks with the following sequence of commands [duplicate]

I have a script that reads a file that contains urls and then prints out the ip addresses. However, I am unable to find resources as to why the pipe blocks when I add the command cut -d" " -f4 at ...
1
vote
1answer
28 views

Is it possible to pipe into dirsplit?

I'm trying to figure out the "Linux way" of accomplishing the following task. I have a directory with a bunch of sub-directories, each of which contain multiple files of various filetypes. I can ...
8
votes
4answers
5k views

Pipe the output of a command if it is successful

INPUT_FILE=`ls -rt $MY_DIR/FILE.*.xml | head -1 | xargs basename` I wanted to execute the second command (head -1) only if the first command is successful. How do I improve this command?
0
votes
0answers
28 views

What does the “pipe” command do in mailx?

When using the mailx command line utility, there is a command called pipe listed in the man page (although not visible when using the help command). What does this command do? I want to be able to ...
2
votes
5answers
3k views

Why does the command shuf file > file leave an empty file, but similar commands do not?

I know this is sort of a duplicate of another question (Why this sort command gives me an empty file?) but I wanted to expand on the question in response to the answers given. The command shuf ...
0
votes
0answers
23 views

How do I connect to Hyper-V VM's serial port from Linux?

I run Linux as a virtual machine on MS Hyper-V hypervisor from Windows Server (2008 R2 and 2012 R2, 2016 would be nice as well) and I want to read output from the VM's serial port (one-way ...
1
vote
0answers
33 views

When was pipelining introduced?

I understand early versions of Unix were licenced from AT&T to Universities and ran on PDP computers. I read in Tanenbaum's book (operating systems) that the first versions of Unix were only a ...
8
votes
1answer
89 views

Why doesn't file redirection to and from a named pipe work, but piping to cat does?

This is a simple echo server in Unix, using nc: mkfifo fifo cat fifo | nc -k -l 4458 -v | cat >fifo (based on this) As I can see it, the data flow works as follows: fifo (my named pipe) | | (...
4
votes
3answers
486 views

How do pipes and infinite streams work?

I usually use command1 | command2 | command3 a lot in Linux but most of them are dealing with definite content. When I tried this with an infinite stream cat | sed '' | sed '' which hopefully ...
16
votes
5answers
963 views

How to trick a command into thinking its output is going to a terminal

Given a command that changes its behaviour when its output is going to a terminal (e.g. produce coloured output), how can that output be redirected in a pipeline while preserving the changed behaviour?...
0
votes
1answer
28 views

how to color specific strings in standard output and standard error [duplicate]

How do you color specific strings in standard output and standard error? For example: gpg --verify Outputs either a good signature or a bad signature string. How can I color the good one, green; ...
1
vote
1answer
44 views

Persistent process substitutions? [duplicate]

Is there a way to take a process substitution like <(grep foo some/file), and save it to a "file-like entity" on disk (reminiscent of a fifo), so that, every time this thing was opened for reading ...
3
votes
2answers
38 views

Mark descriptor as terminal for the -t test

One can test if a descriptor is connected to terminal with -t test. if [ ! -t 0 ]; then echo "Stdin is not from terminal" fi Having a descriptor X, pointing to a file or to a process (with >(...
0
votes
1answer
39 views

globbing and wildcards

Are wildcards ever supposed to be expanded for a program that is not the first in the pipeline? Every example uses wildcard for the first program in the pipelilne. For example, the following does ...
20
votes
4answers
1k views

How to invoke vim editor and pipe output to bash

Sometimes I need to write text and then pipe that text into another command. My usual workflow goes something like this: vim # I edit and save my file as file.txt cat file.txt | pandoc -o file.pdf # ...
1
vote
1answer
48 views

Filter out duplicate lines of tail -f [duplicate]

I often need to tail -f apache access logs for websites to troubleshoot issues- one thing that makes it annoying is that anyone loading a page once may cause 12+ lines to get written to the log, and ...
222
votes
11answers
89k views

Turn off buffering in pipe

I have a script which calls two commands: long_running_command | print_progress The long_running_command prints a progress but I'm unhappy with it. I'm using print_progress to make it more nice (...
2
votes
2answers
140 views

How do I double each line of piped output? [duplicate]

How do I double each line of input piped in? Example: echo "foobar" | MYSTERY_COMMAND foobar foobar
9
votes
4answers
10k views

Output to stdout and at the same time grep into a file

I have a script that outputs text to stdout. I want to see all this output in my terminal, and at the same time I want to filter some lines and save them in a file. Example: $ myscript Line A Line B ...
1
vote
0answers
7 views

Track amount of data application and a device file are exchanging

Is it possible to track how much data is going into a device file and an application as a third person? Background A web server is communicating with a device /dev/ttyACM0. The server offers a HTTP ...
0
votes
2answers
31 views

operate on different sections of stdin without intermediate file

Is there a simple way to do this with redirection or pipes without creating FILE1? I want to apply process2 to the body of the output of process1, without touching the first and last few lines. ...
0
votes
1answer
32 views

How to determine if it is a fault of the Network Card?

Basically, I have a PC having problems. Let's call it POO. POO is a pc running Gentoo Linux, with KDE. It is quite freshly setup on an old hardware. So far I do not notice it has other problems, ...
8
votes
5answers
474 views

Is there a tool to dynamically redirect output to a new file on request

I'm currently redirecting the output of a monitoring tool to a file, however what I'd like to do, is to redirect this output to a new file on my request (using a keybinding), without stopping the said ...
3
votes
2answers
34 views

Chaining Linux terminal commands

I have a function that has my EV3 speak speak(){ espeak -a 200 -s 130 -v la --stdout "$@" | aplay; } it works by simply speak "Say this" I want it to say the contents of a file, so I have this ...
8
votes
3answers
12k views

Why is my variable local one 'while read' loop, but not in another seemingly similar loop?

Why do I get different values for $x from the snippets below? #!/bin/bash x=1 echo fred>junk ; while read var ; do x=55 ; done <junk echo x=$x # x=55 .. I'd expect this result x=1 cat ...
0
votes
1answer
55 views

grep with piping and showing multiple lines

I am trying to use grep to search a large text file, and return the paragraph containing a few key words. I also want to return the surrounding lines in the result. So, for example I have the ...
5
votes
1answer
58 views

Cannot grep jobs list when jobs called in a function

I can grep the output of jobs, and I can grep the output of a function. But why can't I grep the output of jobs when it's in a function? $ # yes, i can grep jobs $ jobs [1]+ Running vim [2]+...