Chains the standard streams of a series of commands

learn more… | top users | synonyms

11
votes
5answers
9k views

how do I trim each line in bash?

I would like to remove all leading and trailing spaces and tabs from each line in an output. Is there a simple tool like trim I could pipe my output into?
14
votes
3answers
7k views

How to check if pipe is empty

I have piped line in bash script and want to check if pipe has data, before feeding program Searching I found about test -t 0 but it doesn't work here. Always returns false. So how to be sure that ...
2
votes
0answers
33 views

BASH: Fifo troubles - seems selective about input

So I am having a problem with a BASH service in Debian 7 that I've been working on for quite a while and that randomly started having trouble with its fifo, or so it seems. It is based on kind of your ...
2
votes
4answers
78 views

Extract two values from the output of a command

I'm using the following command in a script to get synthetic wifi info: echo "$( iw dev wlp1s0 link | grep '^\s*SSID:\s' | sed -r 's/^\s*SSID:\s//' ) $( iw dev wlp1s0 link | grep ...
-1
votes
0answers
34 views

How to merge these three task in a single command ( grep | sed | echo)

I am trying something where I parse out from the tags in a file then add back the HTML. I see everyone suggest not to do such with regEx (grep). but here it is grep -m 1 -oP ...
10
votes
4answers
3k views

Have Bash script wait for status message before continuing

I'm firing up Selenium server with a bash script and as you can see from the timestamps on the log below, it takes about 32 seconds for the thing to fully come online: Feb 28, 2012 10:19:02 PM ...
11
votes
3answers
1k views

Why do some commands not read from their standard input?

I wonder what when we should use pipeline and when we shouldn't. Say for instance, to kill certain process which handling pdf files, the following will not work by using pipeline: ps aux | grep pdf ...
2
votes
1answer
43 views

pipe mplayer output to other tool

I'm trying to play an online live stream via mplayer and pipe the audio from that stream to another tool. Playing the stream works fine: mplayer http://user:pass@192.168.0.1:8080/stream.flv But ...
3
votes
2answers
70 views

How to use a bash function like a regular command in a pipe chain?

I'm performing several commands on large files from within a bash script. In order to monitor the progress I use the pv command. An example command could look like this cat $IN_FILE | pv -w 20 -s ...
1
vote
1answer
18 views

How to preprocess CUPS' `lp` with a script for a specific printer?

(How) Is it possible to preprocess lp automatically with a script? Piping inbetween (fileToPrint | script.sh | lp -d myPrinter) is no option for me, because I have a lot of source code that does lp ...
0
votes
1answer
66 views

Saving from arecord on a raspberry pi across a network to a mac

I've currently got a usb mic plugged into a Raspberry Pi B+ and am using the following code to record data: arecord --buffer-time=5000000 -D plughw:1,0 -f cd -t raw | lame -r - stream.mp3 I then ...
1
vote
3answers
51 views

Is it possible to create dynamic content on file read operation?

Is it possible to generate file content based on file name? I need a lot of similar .conf files with content depending on file name only. Can I create some "dynamic" file and generate a bunch of ...
3
votes
2answers
50 views

When does cat finish reading from a pipe?

Given the following code, why does cat only print the contents of the pipe after I've typed \n or CTRL+D? What are the conditions for cat to actually print what it read? #include <stdio.h> ...
1
vote
2answers
57 views

Pipe to my own script

I've recently noticed this message from grep "grep: warning: GREP_OPTIONS is deprecated; please use an alias or script" I receive this warning because --color=always is defined in my .bashrc file. ...
11
votes
4answers
452 views

Why isn't apt-cache policy output piped?

Cannot get why $ apt-cache policy foo N: Unable to locate package foo but $ apt-cache policy foo 2>&1 | grep . is empty. Where in the latter call am I doing the wrong assumption? The ...
5
votes
3answers
78 views

How to use sed to manipulate continuously streaming output?

I am putting together a presentation for a non-technical audience. I have a program running in bash that outputs a continuous stream of values, a few of which are important. I would like to highlight ...
1
vote
0answers
10 views

If an I/O function cannot fail with EINTR, does that mean it never blocks? [migrated]

For example, consider fdopen. I have been wondering whether it's possible for this to block, if, say, the file descriptor argument refers to a pipe or socket. A Google search hasn't turned up anything ...
2
votes
1answer
65 views

Command for killing specific PID provided by previous command

Sometimes I need to kill a process (the reasons why are not important). And I know I can find that process with the following command: lsof -i :8080, being my candidate the last process in the output ...
1
vote
1answer
44 views

How to create dynamic process substitution based on the input (multiple FIFOs)?

I'm trying to create multiple process substitutions based on the output from the other command, however I'm not sure how I can achieve that. As sample scenario, I've given list of files: touch a.txt ...
1
vote
1answer
91 views

Formatting grep output with awk. Simple case and background case

I'm struggling with this problem. I have this line: mplayer *.* 2>/dev/null | grep Playing which just launchs mplayer and only the line matching Playing filename is printed on screen. This works ...
0
votes
1answer
30 views

Feeding audio from microphone live into python

Well, so I was reading about this raspberry pi tutorial that makes you play audio files from raspberry pi and stream them into a FM signal and something got me curious. You can stream .wav files, ...
-1
votes
1answer
25 views

Does those two constructions do the same thing: proces <input.txt vs cat input.txt | process in Linux

Does those two constructions do the same thing: proces <input.txt vs cat input.txt | process in Linux?
1
vote
1answer
40 views

In-place substitution for paste command [duplicate]

When I run the following code echo '1' > file.txt echo '2' | paste file.txt - > file.txt I would expect the content of file.txt be 1 2, because the paste command as it name suggests should ...
1
vote
0answers
18 views

Correct/Efficient/Reliable way to import log file entries to a MySQL database?

I'd like to be able to query our server access log files to identify abuse patterns. Log files really aren't very queryable, and it would be fairly easy to do what I wanted if each hit was a row in a ...
13
votes
5answers
27k views

Difference between 2>&1 > output.log and 2>&1 | tee output.log

I wanted to know the difference between the following two commands 2>&1 > output.log and 2>&1 | tee output.log I saw one of my colleague use second option to redirect. I know ...
6
votes
1answer
74 views

Does tee slow down pipelines?

I am wondering whether tee slows down pipelines. Writing data to disk is slower than piping it along, after all. Does tee wait with sending data through to the next pipe until it has been written to ...
0
votes
2answers
36 views

Separate execution status of piped commands

In a command where we have lot of pipes redirecting their outputs further, is there any way to get to know the value of echo $? till half of it's execution or till certain number of pipes, as opposed ...
4
votes
1answer
79 views

uniq and bash for loop not writing to stdout before stdin closing (for one-line website visitor notification system)

I'm trying to trigger a beep on the PC speaker for every unique visitor of a website. After some brainstorming, it seemed to be possible with one line: for e in `ssh me@mymachine "tail -n 1 -f ...
2
votes
0answers
35 views

Scripted write-then-read from Unix domain socket using socat

I have a simple server listening on a Unix domain socket (on a CentOS box in case that's important). When I send the command "DATE" to the socket it writes back a welcome message and then "APRIL 15 ...
2
votes
3answers
39 views

Extracting archive by piping contents of archive into the tar command

I am creating a self-extracting archive and I have got the basic thing going. However I want to set the command that extracts the archive contents tail -n+$ARCHIVE_START_LINE $0 into a variable. I ...
14
votes
9answers
520 views

Filter or pipe certain sections of a file

I have an input file with some sections the are demarcated with start and end tags, for example: line A line B @@inline-code-start line X line Y line Z @@inline-code-end line C line D I want to ...
2
votes
1answer
44 views

Syntax When Combining dd and pv

I am a linux newbie; so, sorry if this is obvious to you... sudo dd if=/dev/sda bs=64k | pv --size 1.5t | dd of=/dev/sdb Does the block size for dd go on the left side of this after the input as ...
1
vote
2answers
42 views

Merge lines in file together from regex to “lineify” multiline log log files

Stadard unix command line tools (grep, cut, sed etc.) all work one line at a time. And nearly always, this is great. However I am trying to parse some postgresql slow query logs. Each entry has some ...
0
votes
2answers
49 views

How is pipe involved in ssh?

My internet connection was lost When I ssh to a remote machine over the internet. It says $ Write failed: Broken pipe How is pipe involved in ssh? Does ssh implement communication between the two ...
0
votes
2answers
38 views

Are commands in a pipeline run simultaneously? [duplicate]

In a pipeline such as command1 | command2, are the commands run simultaneously (i.e. have the same lifetime), or command2 starts to run at the time command1 exits, or something else? My question ...
2
votes
1answer
48 views

How to tell xargs which argument to choose ? [duplicate]

When xargs redirects the first commands output into the second command's argument and there is no choice about which argument for which element of the output, then there is only one way, for instance: ...
16
votes
4answers
601 views

How can I implement a circular flow of data among interconnected commands?

I know of two types how commands can be connected with each other: by using a Pipe (putting std-output into std-input of the next command). by using a Tee (splice the output into many ...
4
votes
1answer
89 views

How to perform the watch command onto expression with pipes?

I learned today the wonderful shuf command: ls | shuf shows me a listing of the working direcotry, but thanks to shuf each time I execute this piped command expression with another order. So I ...
1
vote
2answers
54 views

pipe in shell with redirection

My understanding is that a pipe in shell like a | b redirects b's stdin to a's stdout. However, I tried ls | less < somefile, the behavior is very strange. less shows the output of ls concatenated ...
0
votes
1answer
32 views

Is it possible to link file to an output of command line? [duplicate]

I want to create symlink, or a pipe, or something else that could be used in lieu of a file. I want that pseudo-file to execute a command and output of that command should become the contents of that ...
3
votes
4answers
615 views

tar a result of wget

I want to wget a file and tar it, in one command, I guess it is simple but I can't get it done. I tried several things. wget <url> | tar -cvz file.gz.tar - tar -cvzf file.tar `wget ...
1
vote
1answer
44 views

Only pipe on Success [duplicate]

Is there a way in the Linux command line, to only have a command pipe its output to another, when the command didn't fail? $ cmda | cmdb I would like it, so when cmda returns a status code other ...
3
votes
1answer
39 views

why does the output of xargs -n2 applied to the same number of arguments different whether piped or from keyboard?

when I execute xargs -n2, enter x x x x x and hit Enter, I get only 4 x back: $ xargs -n2 x x x x x x x x x But, when I pipe the x x x x x into the same command, I get the same amount of x back: ...
1
vote
1answer
52 views

How does 'bash' or 'gnome-terminal' print user and path information? [closed]

I am wondering if there is a different mechanism between printing user and path information(e.g. "guest:~/home/guest$") and command outputs. When I try to PIPE stdin and stdout of "bash", I can only ...
47
votes
4answers
45k views

How big is the pipe buffer?

As a comment in I'm confused as to why "| true" in a makefile has the same effect as "|| true" user cjm wrote: Another reason to avoid | true is that if the command ...
0
votes
1answer
53 views

Can a command in a shell pipeline determine the tty characteristics of its context?

I have a bash script that columnizes a list of items. By default it will guess at the number of columns to output, based on the terminal width as reported by stty size. But when the script is in a ...
4
votes
4answers
364 views

How to fill a file with a stream from /dev/urandom with a specified number of lines ?

I am trying to fill a file with a sequence of random 0 and 1s with a user-defined number of lines and number of characters per line. the first step is to get a random stream of 0 and 1s: cat ...
47
votes
4answers
5k views

In what order do piped commands run?

I've never really thought about how the shell actually executes piped commands. I've always been told that the "stdout of one program gets piped into the stdin of another," as a way of thinking about ...
3
votes
1answer
71 views

rlwrap -z pipeto: piping output through pagers

I am using rlwrap to colorize prompt in asterisk CLI: rlwrap -s 99999 -a -pRED /usr/sbin/asterisk -r I read in man rlwrap that I can also use rlwrap -z pipeto to pipe output through a ...
1
vote
1answer
47 views

Using SMB to pipe contents of file to local command on local machine

Here is my situation. I have a 100gig file on a remote server, what I need to do is connect to that machine and zcat that file and pipe the output of zcat to a command on a local machine... I was ...