I have a file that contains the paths of various files.

files.txt :


I want to use it in a script to process the files.

cat files.txt | xargs -P 12 -L 1 -I % /software/Linux/itt/idl/idl81/bin/idl -e program -args %

I basically want to send these paths as a string to this IDL program via xargs as an argument.

However when I use cat, it actually cats file1.txt, file2.txt, and outputs their content directly as an argument into this IDL program.

How can I get it to just send each and every line from this files.txt as a string to this IDL script?

  • 2
    seems to work here; are you doing something like cat file* | xargs ...?
    – Jeff Schaller
    Oct 15, 2015 at 17:17

2 Answers 2


You could use a small script with a loop like this

Let it be runIDL.sh

while (read filepath)
    /software/Linux/itt/idl/idl81/bin/idl -e program -args "$filepath"
done < "$1"

It's simple, elegant and should work like a charm. Just run ./runIDL files.txt and that's it!


GNU Parallel may work for you:

cat files.txt | parallel /software/Linux/itt/idl/idl81/bin/idl -e program -args {}

It will default to 1 process per CPU core and it will make sure the output of two parallel jobs will not be mixed.

GNU Parallel is a general parallelizer and makes is easy to run jobs in parallel on the same machine or on multiple machines you have ssh access to. It can often replace a for loop.

If you have 32 different jobs you want to run on 4 CPUs, a straight forward way to parallelize is to run 8 jobs on each CPU:

Simple scheduling

GNU Parallel instead spawns a new process when one finishes - keeping the CPUs active and thus saving time:

GNU Parallel scheduling


If GNU Parallel is not packaged for your distribution, you can do a personal installation, which does not require root access. It can be done in 10 seconds by doing this:

(wget -O - pi.dk/3 || curl pi.dk/3/ || fetch -o - http://pi.dk/3) | bash

For other installation options see http://git.savannah.gnu.org/cgit/parallel.git/tree/README

Learn more

See more examples: http://www.gnu.org/software/parallel/man.html

Watch the intro videos: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial: http://www.gnu.org/software/parallel/parallel_tutorial.html

Sign up for the email list to get support: https://lists.gnu.org/mailman/listinfo/parallel

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .