Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to run a java command once for every match of ls | grep pattern -. In this case, I think I could do find pattern -exec java MyProg '{}' \; but I'm curious about the general case - is there an easy way to say "run a command once for every line of standard input"? (In fish or bash.)

share|improve this question
up vote 41 down vote accepted

That's what xargs does.

... | xargs command
share|improve this answer
Not quite. printf "foo bar\nbaz bat" | xargs echo whee will yield whee foo bar baz bat. Maybe add the -L or -n options? – Jander Feb 17 '11 at 5:17
@Jander The question was rather general, so I gave the general tool. True, you will have to adjust its behavior with options depending on the specific circumstances. – Keith Feb 17 '11 at 5:25
... | tr '\n' '\0' | xargs -0 – Vardhan Feb 17 '11 at 11:12
like, "the specific circumstances which give the right answer to the question". :) – mattdm Feb 17 '11 at 18:17
If you want to see the proper way to do this with xargs, see my answer below. – Michael Goldshteyn Jun 30 '15 at 16:08

In Bash or any other Bourne-style shell (ash, ksh, zsh, …):

while read -r line; do command "$line"; done

read -r reads a single line from standard input (read without -r interprets backslashes, you don't want that). Thus you can do either of the following:

$ command | while read -r line; do command "$line"; done  

$ while read -r line; do command "$line"; done <file
share|improve this answer
When I tried tail -f syslog | grep -e something -e somethingelse| while read line; do echo $line; done it didn't work. It worked with a file piped into the while loop, worked with just the tail -f, worked with just grep, but not with both pipes. Giving the grep the --line-buffered option made it work – user16468 Mar 10 '12 at 20:39

The accepted answer has the right idea, but the key is to pass xargs the -n1 switch, which means "Execute the command once per line of output:"

cat file... | xargs -n1 command

Or, for a single input file you can avoid the pipe from cat entirely and just go with:

<file xargs -n1 command
share|improve this answer
Also of interest is the ability of xargs to not run if stdin is empty: --no-run-if-empty -r : If the standard input does not contain any nonblanks, do not run the command. Normally, the command is run once even if there is no input. This option is a GNU extension. – Ronan Jouchet Oct 24 '15 at 19:58
This should be the accepted answer :) – robert Feb 3 at 21:35
How dyou access the line inside command ? – B T Apr 14 at 1:21

I agree with Keith, xargs is the most general tool for the job.

I usually use a 3 step approach.

  • do the basic stuff until you have something you would like to work with
  • prepare the line with awk so it gets the correct syntax
  • then let xargs execute it, maybe with the help of bash.

There are smaller and faster ways, but this ways almost always works.

A simple example:

ls | 
grep xls | 
awk '{print "MyJavaProg --arg1 42 --arg2 "$1"\0"}' | 
xargs -0 bash -c

the 2 first lines selects some files to work with, then awk prepares a nice string with a command to execute and some arguments and $1 is the first column input from the pipe. And finally I make sure that xargs sends this string to bash that just execute it.

It is a little bit overkill, but this recipe has helped me in a lot of places since it is very flexible.

share|improve this answer
Note, xargs -0 uses the null byte as a record separator, so your awk print statement should be printf("MyJavaProg --args \"%s\"\0",$1) – glenn jackman Feb 17 '11 at 14:30
@glenn: Missed the null char, will update the answer – Johan Feb 18 '11 at 12:43
@Johan not a big deal, but if you're using awk you can have it do the pattern match and skip the grep e.g., ls | awk '/xls/ {print... – Eric Renouf May 31 at 15:19

GNU Parallel is made for that kind of tasks. The simplest usage is:

cat stuff | grep pattern | parallel java MyProg

Watch the intro video to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ

share|improve this answer
No real need for the cat here since grep can directly read the file – Eric Renouf May 31 at 15:20
Thanks for the link, I don't necessarily agree that it's easier to read, but nice to know it was considered regardless. I would only now slightly quibble that the link doesn't actually apply here since the alternative is not really < stuff grep pattern but is grep pattern stuff with no redirection or cat required at all. Still, that doesn't materially change your argument and if you think it's clearer to always use things in a pipe that starts with cat, then power to you – Eric Renouf Jun 1 at 0:32

When dealing with potentially unsanitized inputs, I like to see the entire job 'spelled out' line by line for visual inspection before I run it (especially when it's something destructive like cleaning people's mailbox's).

So what I do is generate a list of parameters (ie. usernames), feed it to a file in one-record-per-line fashion, like this:


Then I open the list in vim, and mangle it with search and replace expressions until I get a list of full commands that need to get executed, like this:

/bin/rm -fr /home/johndoe  
/bin/rm -fr /home/jamessmith 

This way if your regex is incomplete, you will see in what command will have potential problems (ie. /bin/rm -fr johnnyo connor). This way you can undo your regex, and try it again with a more reliable version of it. Name mangling is notorious for this, because it's hard to take care of all the edge cases like Van Gogh, O'Connors, St. Clair, Smith-Wesson.

Having set hlsearch is useful for doing this in vim, as it will highlight all the matches, so you can easily spot if it doesn't match, or matches in an unintended way.

Once your regex is perfect and it catches all the cases you can test for/think of, then I usually convert it to a sed expression so it can be fully automated for another run.

For cases where the number of lines of input prevents you from doing a visual inspection, I highly recommend echoing the command to the screen (or better yet, a log) before it executes, so if it errors out, you know exactly which command caused it to fail. Then you can go back to your original regex and adjust once more.

share|improve this answer

Also, while read loop in fish shell (I assume you want fish shell, considering you used tag).

command | while read line
    command $line

Few points to note.

  • read doesn't take -r argument, and it doesn't interpret your backslashes, in order to make most common use case easy.
  • You don't need to quote $line, as unlike bash, fish doesn't separate variables by spaces.
  • command by itself is a syntax error (to catch such use of placeholder arguments). Replace it with the real command.
share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.