Chains the standard streams of a series of commands

learn more… | top users | synonyms

0
votes
1answer
48 views

Piped search queries in vim or less

I would like to be able to search a log file for lines that being with ^2014-02-18 15:30:[0-9:,]* and have UserName:someguy much later in the same line. I am sure I could do a regex to include any ...
3
votes
5answers
768 views

Counting files in folder without wc

Why does the following scripts give a count of 0 instead of giving the count of files present in the directory? #!/bin/bash cd /root/Jamshed/script count=0 ls -lrt > all_files cat all_files | ...
2
votes
1answer
647 views

How can I pipe commands to any terminal?

I often use more than one terminal (or terminal emulator) at a time; and while in X I can copy-paste commands, besides not being very practical, it obviously does not work on the real TTY. The first ...
4
votes
2answers
2k views

How to compose bash functions using pipes?

I have few functions defined in this fashion: function f { read and process $1 ... echo $result } I want to compose them together so that invocation would look like f | g | h. What idiom ...
1
vote
3answers
149 views

How to use the pipe command to open a directory after find?

Is there a way to opend found directories (via find) all in one command? One of the things I tried to study several times and never been able to really get, is the effective use of pipe: is it maybe ...
4
votes
3answers
744 views

Please help explain this bash output redirection

This script was posted as answer to a Question. And I'm trying to work out what's going on. result=$( { { ssh host app-status >&3 3>&-; echo "$?" } | { until read ...
6
votes
3answers
573 views

How to modify output in bash command pipeline

For example, I got from some command some lines $ some-command John Bob Lucy Now I'd like to add chaining command, that modifies output. $ some-command | other-command Hi John Bye Hi Bob Bye Hi ...
3
votes
2answers
105 views

Repeated commands in a bash pipeline

Sometimes commands repeat in pipeline command. For example (just to illustrate): $ grep -lF 'pattern' ./foo/**/*.bar | xargs dirname | xargs dirname Is there a way to shorten chaining command? For ...
3
votes
4answers
1k views

Pipe all input and output of terminal to file

I do not know if this is even possible or not, but in an attempt to keep track of many changes I am about to make to my system I want to track all input I type and all output from a terminal I'm ...
1
vote
2answers
786 views

Why does this ssh command not work? (cat ./file | ssh <user@host> “cat | sudo -t -t 'cat > /root/location/file'”)

I'm trying to do the following but it's not quite right. cat ~/Desktop/blah.png | ssh -t -t trusktr@50.116.4.56 "cat | sudo -i 'cat | /path/to/blah.png'" I have my public ssh key in the ...
3
votes
1answer
77 views

How can the order of execution in the shell as it relates to redirection be upset about 1 time out of a 1000 on my system? [duplicate]

Considering this Q&A about the order of execution in the shell as it relates to redirection, and despite the fact that if a file doesn't exist it gets created first so that cat example.txt | shuf ...
0
votes
4answers
382 views

Why does the command shuf file > file leave an empty file?

I know this is sort of a duplicate of another question (Why this sort command gives me an empty file?) but I wanted to expand on the question in response to the answers given. The command shuf ...
0
votes
2answers
44 views

How can I use tee to print the input stream?

I have a pipe line that looks like this: cat myfile | awk '$3 > 0 { print $1, $2 * $3 }' | tee >(printf) | perl myscript.pl I would like to print the input to my Perl script using tee. How ...
4
votes
1answer
71 views

Shell one-liner for adding a line in the middle of a pipe

I write a script which sends the input file to CPP, then to grep, then to a certain program: cpp $1 | grep -v "#" | theprogram But, "theprogram" requires the input to start with a certain constant ...
0
votes
0answers
85 views

Postfix mail how to debug transmission to mail handling agent

I have a postfix server running on Ubuntu (and Debian previously). The server is configured to read users and groups from an OpenLDAP server and delivers mail first the spam filter and then to a ...
3
votes
2answers
631 views

grep string from contents of file command then delete file

I have a peculiar problem I need help with. I have a large batch of .docx files where a certain number of them are junk. I know how to identify these files from the file command, as they contain the ...
2
votes
1answer
197 views

Piping periodic data between Python scripts

How to receive data from the environment e.g. bash I am trying to launch two Python scripts in bash and pipe stdout from one of them to stdin of the other. The first script outputs a single ...
2
votes
3answers
183 views

Preserving color output with cut

Using bash interactively I am trying to use cut to prevent the lines from going longer than my terminal width. But when using a command with colorized output all the color gets removed by cut. For ...
1
vote
1answer
53 views

Memory-efficient utility to return N first sorted values

I would like to implement a very popular MapReduce example using only existing programs operating in a UNIX way. The problem is to find N most frequent values in an enormous amount of data. The ...
3
votes
1answer
244 views

Why does grep change the length of output lines?

I often use dpkg or aptitude combined with grep when I want to list certain packages available or installed on my system, but I noticed that when I add | grep, the output lines look a little bit ...
1
vote
5answers
5k views

Pass the output of previous command to next as an argument

I've a command that outputs data to stdout (command1 -p=aaa -v=bbb -i=4). The output line can have the following value: rate (10%) - name: value - 10Kbps I want to grep that output in order to ...
2
votes
2answers
117 views

Piping to the command substitution of a string containing pipes

This works— $ x='grep a' $ echo ab | $x ab This doesn't— $ x='grep a | grep b' $ echo ab | $x grep: |: No such file or directory grep: grep: No such file or directory grep: b: No such file or ...
1
vote
2answers
423 views

Help me understand this echo/spawn/send/expect construct

Here is a line from a bash script I am trying to understand: echo "spawn myscript.sh arg1 arg2; expect \"Please enter your value: \"; send \"myval\r\"; expect eof" | expect I think I understand the ...
4
votes
3answers
177 views

processing command output line by line, without mixing standard input

So, I have a problem very similar to Bash: How to read one line at a time from output of a command?, I am doing: while read path; do mplayer $path; done < <(find-me-random-mp3s) or ...
1
vote
1answer
47 views

Mysql import the most recent file from a directory

I have a directory with daily backups of my entire MySQL database. I want to import the most recent backup into the database. I know to import a backup I need to use mysql -u root -ppasswordhere ...
3
votes
1answer
266 views

How does the piping in this command ultimately achieve to kill the process?

I have seen the following line in a bash script for killing a process(in this case started with the command loadgen): ps xww | grep -i "loadgen" | grep "PATTERNMATCH_FACT.xml" | cut -c1-5 | xargs -i ...
4
votes
2answers
1k views

Is there any way to use xargs across a pipe?

I'm trying to automatically convert flac files moved to a folder into mp3 in another folder. My current line of code is this: inotifywait -m -r -q -e moved_to --format "'%w%f'" ~/test | xargs -I x ...
5
votes
1answer
172 views

Replace CD number in MP3 ID3 tag

I have a library of MP3 files where in some files the "CD number" tag (TPOS) contains "CDx" values, e.g. "CD1" or "CD2". The ID3 spec says that TPOS values shall be numerical strings. Processing these ...
16
votes
9answers
1k views

Grep lines but let the first line through [duplicate]

Consider the following output from df. Filesystem Size Used Avail Use% Mounted on /dev/sda1 23G 6.1G 16G 29% / udev 10M 0 10M 0% /dev tmpfs 397M 420K ...
2
votes
3answers
109 views

The processing order and printing of files with find

This simple BASH script collects msgstrs in a PHP project: #!/bin/sh TARGET=/path/to/project LPATH=locale/cy_GB/LC_MESSAGES POFILE=messages.po mkdir -p $LPATH touch $TARGET/$LPATH/$POFILE cd $TARGET ...
24
votes
3answers
589 views

Multicolored Grep

I'm trying to get each grep command to highlight it's results in a different color. I can do it manually with a line like this: ls -l GREP_COLORS='mt=01;32' grep c | GREP_COLORS='mt=01;31' grep o | ...
14
votes
2answers
452 views

Use of ^ as a shell metacharacter

I wrote a small script today which contained grep -q ^local0 /etc/syslog.conf During review, a coworker suggested that ^local0 be quoted because ^ means "pipe" in the Bourne shell. Surprised by ...
3
votes
3answers
98 views

Print a message from the left-hand side of a pipe

I want to use a variable from the main shell in a sub shell. I did: export mysql_root_password="test" ( echo $mysql_root_password ) | dialog --gauge "Working hard..." 6 50 There is no output. ...
4
votes
3answers
189 views

cycle through one command's output and use in another command [duplicate]

How can I use the output of one command - line-by-line - into another command? I know how to do this with the | pipe symbol, but this uses the entire output in one command. I would like to go line by ...
1
vote
3answers
101 views

Automating pipes?

I find myself using basically the same line over and over again: cat file | command1 | command2 | command3 > file Is there a way I can put all these pipes into one script, so I can just run ...
6
votes
2answers
346 views

Piping data to a process's stdin without causing EOF afterward

I have an executable that starts a user-interactive shell. I would like to, upon launch of the shell, inject a few commands first, then allow the user to have their interactive session. I can do this ...
5
votes
2answers
595 views

Obtaining PID of command earlier in Pipeline

I'm writing a bash script to use inotifywait to monitor a directory and kick off actions when changes are detected. Something like: inotifywait -m ... | while read f; do something; done Since ...
2
votes
2answers
175 views

turn off buffering for `hping` in OpenBSD

It would appear that the output from hping is fully buffered when piped to perl for further line-by-line processing, so piping hping to perl doesn't work. hping --icmp-ts example.ca | perl -ne 'if ...
6
votes
4answers
2k views

how do I trim each line in bash?

I would like to remove all leading and trailing spaces and tabs from each line in an output. Is there a simple tool like trim I could pipe my output into?
2
votes
3answers
231 views

How can I tell echo/printf in Bash to swallow escape codes, based on a condition?

I have a Bash script that is being run interactively and as a cron job. When run interactively it outputs colored text on the terminal. However, when run in cron it doesn't have a terminal and ...
3
votes
4answers
152 views

saving output of another command

I need to select certain lines from the log file and also save them to a text file. I tried the following but none of them are working as expected. The file "todel.txt" shows 0 bytes. tail -f ...
2
votes
2answers
308 views

Pipe command to tail: when is the first command aborted?

In this example: $ for i in {1..3}; do sleep 1; echo $i; done | head -n 2 why is the first command (for loop) killed only just before the 3 is displayed ? I expected it to be killed right after ...
2
votes
1answer
130 views

Appending to same array in various loops, only last values remain Bash 4

I have something like this: FILES=() for i in *.map do FILES+=($i) done find /var/candy -name "chocolate_[0-9]" | while read snack do FILES+=($snack) done for file in ...
1
vote
1answer
140 views

`cat /dev/ttyACM` loses data when piped through netcat

I'm reading data from /dev/ttyACM0 using cat /dev/ttyACM0 on a Raspberry Pi. The data comes from a device, which sends lines in the form 15.00 1316013 15 where the first number is a value, the ...
2
votes
1answer
608 views

Linux nuisance: /dev/stdin doesn't work with sockets

Linux has this blitheringly annoying peculiarity that /dev/stdin won't work with sockets - it is hardcoded to return ENXIO. Try this: socat TCP-OPEN:localhost:1234 EXEC:cat\ /dev/stdin,nofork ...
2
votes
1answer
345 views

Piping Sox and FFMPEG together

I'm processing a variety of audio files in a bunch of different formats and I'd like to unify their format and configuration using FFMPEG and SoX. There are two steps to my process: Convert the ...
22
votes
5answers
2k views

Pipe to multiple files in the shell

I have an application which will produce a large amount of data which I do not wish to store onto the disk. The application mostly outputs data which I do not wish to use, but a set of useful ...
1
vote
1answer
150 views

Pipe input to another process one line at a time

I have a script which can take one file name at a time and I want to run it for many files in a folder. So, I want to pipe the output of: find . -iname '*.m' to my script but one line at a time. ...
2
votes
3answers
557 views

Running tcpdump, tee and scp

I'm trying to run the following: $ tcpdump -l -X -i eth1 -n tcp port <port> | tee capture.txt | \ scp capture.txt root@10.3.3.227:/home/checker/ tcpdump -l -X -i eth1 -n tcp port ...
13
votes
4answers
277 views

pipe, { list; } only works with some programs

Need explanations from power users for such unpredictable behaviour: ps -eF | { head -n 1;grep worker; } UID PID PPID C SZ RSS PSR STIME TTY TIME CMD root 441 2 0 ...