Pass multiple command line arguments to an executable with text files

Question

I have a C executable that takes in 4 command line arguments.

program <arg1> <arg2> <arg3> <arg4>

I'd like to create a shell script that continually runs the executable with arguments supplied by text files. The idea would be something like this:

./program "$(< arg1.txt)" "$(< arg2.txt)" "$(< arg3.txt)" "$(< arg4.txt)"

Where the arguments supplied for run n would be on line n of each of the files. When I tried doing this, the printf() calls were interfering with each other or some other funny business was going on. I would also be open to a script that takes only one file where the arguments are delimited in some way.

Answer 1

while 
  IFS= read -r a1 <&3 &&
    IFS= read -r a2 <&4 &&
    IFS= read -r a3 <&5 &&
    IFS= read -r a4 <&6
do
  ./program "$a1" "$a2" "$a3" "$a4" 3<&- 4<&- 5<&- 6<&-
done 3< arg1.txt 4< arg2.txt 5< arg3.txt 6< arg4.txt

That runs the loop until one of the files is exhausted. Replace the &&s with ||s to run it until all the files are exhausted instead (using empty arguments for shorter files).

With GNU xargs, you could also do:

paste -d '\n' arg[1-4].txt | xargs -n 4 -r -d '\n' ./program

(though beware ./program's stdin would then be /dev/null)

Answer 2

Check out a combination of the sed command and the paste command. First, use paste to create a new stream consisting of all four files. Then use sed to pick the line you want:

paste arg1.txt arg2.txt arg3.txt arg4.txt | sed -n 10p

will print line 10 of the output from paste. This can be fed to the program as arguments using xargs:

paste arg1.txt arg2.txt arg3.txt arg4.txt | sed -n 10p | xargs ./program

If you want to loop over the lines, you can use the seq command to generate a sequence, and then iterate over each value in the sequence:

for i in $(seq 1 100); do 
    paste arg1.txt arg2.txt arg3.txt arg4.txt | sed -n ${i}p | xargs ./program
done

This loop will be slow since it calls paste once for each iteration. It's probably a good idea to create a temp file first.

Answer 3

The best option is to pre-read the files and store the organised results in either arrays or temporary files. Otherwise you will have to call the reading functions four times for each iteration and have the function read further and further afield in the files. This is far from optimal.

This is a temp-files version. Caution, it's not been tested.

PROG=./program
TEMPDIR=tmp
mkdir "$TEMPDIR"

# Create the temp files.
for arg in arg*.txt; do
    i=0
    while read a; do
        ((i++))
        printf "%s\n" "$a" >> "$TEMPDIR"/"$i"
    done < "$arg"
done

# Now the temp files are ready.
## Each file contains all arguments for 1 run of ./program,
## each of them on a separate line.

# Start executing the ./program.
for iteration in "$(ls "$TEMPDIR" | sort -n)"; do
    unset args
    while read arg; do
        args=( "$arg" )
    done < "$TEMPDIR"/"$iteration"
    "$PROG" "${args[@]}"
done

# Finally, remove the temp files.
rm -r "$TEMPDIR"

Answer 4

In Bash (or some other shell that supports $'' and arrays), this should do, as long your input files don't contain tabs^(*):

IFS=$'\t'
paste arg1.txt arg2.txt | 
while read -r -a A ; do 
   [ "${#A[@]}" -eq 2 ] && printf "%s - %s\n" ${A[@]}
done 

paste sticks the input files together line-by-line, read -a A reads the columns to the array A, using tab (from IFS) as a separator. [ "${#A[@]}" -eq 2 ] checks that the array has exactly two members, and ${A[@]} drops them on the command line of printf. Change the command as required.

_{(* if you need to support tabs, I'd move to using e.g. Perl)}

With these input files:

$ cat arg1.txt 
foo bar
doo
$ cat arg2.txt 
one
two two
three three three

The output from the above snippet is:

foo bar - one
doo - two two

The last line from arg2.txt is ignored, since arg1.txt doesn't have a corresponding line. read ignores leading tabs making it impossible to use if we care about which columns had the missing elements.