I have a CSV with 4 fields (id1,id2,id3,filename) like this:


There is a common URL endpoint for the images like


The last column is the unique portion of the URL to the image.

I'm trying to download each image, and name of each image is based on the first 3 fields:

id1 is always 1 digit
id2 is really 5 digits with prepending zeros (so 2314 needs to become 02314)
id3 is really 4 digits (78 becomes 0078),

so in total a 10 digit code. So the first image name should be 2023140078.jpg

If I were to download a single file manually, I'd do something like this:

wget -O 2023140078.jpg http://example.com/assets/?mediafile=/image/1289.jpg

Also if I could write back some sort of status or error code to the row or write an error log, would be great.

2 Answers 2

awk -F, '{printf("%d%.5d%.4d %s\n",$1,$2,$3,$4)}' input.csv | while read newfile filename 
wget -O "${newfile}.jpg" http://example.com/assets/?mediafile=${filename} 
  • After running this for 200,000 rows I realized that not all filenames are unique, and some IDs have several images, how can I fix this?
    – zevnyc
    Jan 17, 2017 at 20:00
  • what you mean by its not unique ? are you ok to add the incremental value in the filename ? eg: 2023140078_1.jpg 2023140078_2.jpg ..... 2023140078_200000.jpg
    – Kamaraj
    Jan 18, 2017 at 8:28
  • Some of the entries have several URLs for the same filename so wget is downloading them all, replacing the first file with the next. Most have only 1 file, but some have several. I'd like for the 1st to stay in the original format and the rest add _1 etc.
    – zevnyc
    Jan 18, 2017 at 10:25

You can set the IFS to a comma and use a while loop to read the 4 variables that uses the file as input:

while read c1 c2 c3 c4; do
  wget -O "$(printf "%d%05d%04d.jpg" "$c1" "$c2" "$c3")" "http://example.com/assets/?mediafile=${c4}"
done < file.txt
  • if second column has the value of 12345, then the filename will be created with 11 digit
    – Kamaraj
    Jan 17, 2017 at 6:30
  • Sorry, I misread your padding requirements. You can use printf command substitution for that: $ IFS=','; while read c1 c2 c3 c4; do echo wget -O "${c1}$(printf %05d ${c2})$(printf %04d ${c3})" "example.com/assets/?mediafile=${c4}"; done < file.txt
    – Kyle
    Jan 17, 2017 at 6:31
  • You should edit your answer to improve it.  Comments are often overlooked and sometimes deleted. And you don't need two printfs; just do $(printf "%d%05d%04d.jpg" "$c1" "$c2" "$c3"). Jan 17, 2017 at 6:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .