I would like to run:

./a.out < x.dat > x.ans

for each *.dat file in the directory A.

Sure, it could be done by bash/python/whatsoever script, but I like to write sexy one-liner. All I could reach is (still without any stdout):

ls A/*.dat | xargs -I file -a file ./a.out

But -a in xargs doesn't understand replace-str 'file'.

Thank you for help.

  • In addition to the other good answers here, you can use the -a option of GNU xargs. – James Youngman Dec 9 '13 at 23:22

First of all, do not use ls output as a file list. Use shell expansion or find. See below for potential consequences of ls+xargs misuse and an example of proper xargs usage.

1. Simple way: for loop

If you want to process just the files under A/, then a simple for loop should be enough:

for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done

2.pre1 Why not   ls | xargs ?

Here's an example of how bad things may turn if you use ls with xargs for the job. Consider a following scenario:

  • first, let's create some empty files:

    $ touch A/mypreciousfile.dat\ with\ junk\ at\ the\ end.dat
    $ touch A/mypreciousfile.dat
    $ touch A/mypreciousfile.dat.ans
  • see the files and that they contain nothing:

    $ ls -1 A/
    mypreciousfile.dat with junk at the end.dat
    $ cat A/*
  • run a magic command using xargs:

    $ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"
  • the result:

    $ cat A/mypreciousfile.dat
    TRICKED with junk at the end.dat.ans
    $ cat A/mypreciousfile.dat.ans

So you've just managed to overwrite both mypreciousfile.dat and mypreciousfile.dat.ans. If there were any content in those files, it'd have been erased.

2. Using  xargs : the proper way with  find 

If you'd like to insist on using xargs, use -0 (null-terminated names) :

find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'

Notice two things:

  1. this way you'll create files with .dat.ans ending;
  2. this will break if some file name contains a quote sign (").

Both issues can be solved by different way of shell invocation:

find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'

3. All done within find ... -exec

 find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' \;

This, again, produces .dat.ans files and will break if file names contain ". To go about that, use bash and change the way it is invoked:

 find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} \;
  • +1 for mentionning not to parse the output of ls. – rahmu Oct 7 '11 at 9:31
  • 2
    Option 2 breaks when the filenames contains ". – thiton Oct 7 '11 at 11:30
  • 2
    Very good point, thanks! I'll update accordingly. – rozcietrzewiacz Oct 7 '11 at 11:39
  • I want just mention, that if zsh is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivial for file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done works in all cases. – jofel Mar 20 '12 at 19:56
  • -1 because I'm trying to figure out how to do xargs with stdin and my question has nothing to do with files or find. – Mehrdad May 7 at 0:04

Try doing something like this (syntax may vary a bit depending on the shell you use):

$ for i in $(find A/ -name \*.dat); do ./a.out < ${i} > ${i%.dat}.ans; done

  • That would not work. It would try to operate on stuff like somefile.dat.dat and redirect all output to a single file. – rozcietrzewiacz Oct 7 '11 at 8:54
  • You're right. I edited the solution to correct it. – rahmu Oct 7 '11 at 8:57
  • OK - Almost good :) Just somefile.dat.ans output stuff would look not so nice. – rozcietrzewiacz Oct 7 '11 at 9:07
  • 1
    Edited! I did not know about '%'. It works like a charm, thanks for the tip. – rahmu Oct 7 '11 at 9:23
  • 1
    Adding a -type file would be nice (can't < directory), and this makes the unusual-filename-fairy sad. – Mat Oct 7 '11 at 9:57

For simple patterns, the for loop is appropriate:

for file in A/*.dat; do
    ./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!

For more complex cases where you need another utility to list the files (zsh or bash 4 have powerful enough patterns that you rarely need find, but if you want to stay within POSIX shell or use fast shell like dash, you will need find for anything non-trivial), while read is most appropriate:

find A -name '*.dat' -print | while IFS= read -r file; do
   ./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!

This will handle spaces, because read is (by default) line-oriented. It will not handle newlines and it will not handle backslashes, because by default it interprets escape sequences (that actually allows you to pass in a newline, but find can't generate that format). Many shells have -0 option to read so in those you can handle all characters, but unfortunately it's not POSIX.


Use GNU Parallel:

parallel ./a.out "<{} >{.}.ans" ::: A/*.dat

Added bonus: You get the processing done in parallel.

Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ


I think you need at least a shell invocation in the xargs:

ls A/*.dat | xargs -I file sh -c "./a.out < file > file.ans"

Edit: It should be noted that this approach does not work when the filenames contain whitespace. Can't work. Even if you used find -0 and xargs -0 to make xargs understand the spaces correctly, the -c shell call would croak on them. However, the OP explicitely asked for an xargs solution, and this is the best xargs solution I came up with. If whitespace in filenames might be an issue, use find -exec or a shell loop.

  • Have a look at why it is a bad idea to parse ls output. – rozcietrzewiacz Oct 7 '11 at 9:13
  • @rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway. – thiton Oct 7 '11 at 9:19
  • 1
    You are wrong about the shell expansion. ls outputs things like spaces without escaping and that is the problem. – rozcietrzewiacz Oct 7 '11 at 9:25
  • @rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks. – thiton Oct 7 '11 at 10:19
  • Yes. You see now, parsing ls is no good. But xargs can be safely used with find - see the suggestion No 2 in my answer. – rozcietrzewiacz Oct 7 '11 at 10:53

There's no need to complicate it. You could do it with a for loop:

for file in A/*.dat; do
  ./a.out < "$file" >"${file%.dat}.ans"

the ${file%.dat}.ans bit will remove the .dat filename suffix from the filename in $file and instead add .ans to the end.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.