Can't make awk loop over file values

Question

Hi guys

I'm trying to loop over the values of a file (or multiple files) passed as a parameter to the script as in, ./myscript file1 file2 ... but awk only recognizes the first line of each file.

#!/bin/bash

while [[ ${#} > 0 ]]
do
 for archivo in ${@}
 do
  read url filename <<<$(awk -F',' '{print $1, $2}' $archivo)
  curl -o $filename $url
 done
done

Files passed as params to the script have the following structure:

url1,filename1
url2,filename2
...

The goal is, whenever the script is called with one or more parameters, awk should parse (and pass) the urls and filenames contained inside each file to curl for it to download the files listed. Any suggestions on how to make it work?

Thanks in advance and happy day!

Answer 1

Aside from the fact that your loop structure appears to be screwed up (you only read one url-filename pair for each file), the unquoted command substitution $(awk -F',' '{print $1, $2}' $archivo) is going to be subject to word-splitting on the default IFS, with the result that the awk output is going to get concatenated onto a single line. Compare:

$ echo $(printf 'foo bar\nbaz bam\n')
foo bar baz bam

versus

$ echo "$(printf 'foo bar\nbaz bam\n')"
foo bar
baz bam

However this is really a useless-use-of-awk - you can set the shell's IFS to , and read your files directly. Probably what you want is:

for archivo in "$@"
do
  while IFS=, read -r url filename rest
  do 
    curl -o "$filename" "$url"
  done < "$archivo"
done