1

I have a CSV with 4 fields (id1,id2,id3,filename) like this:

2,2314,78,/image/1289.jpg
1,19825,179,/image/01288.jpg
2,2585,2281,/image/01287.jpg
3,35879,82,/image/01286.jpg

There is a common URL endpoint for the images like

http://example.com/assets/?mediafile=

The last column is the unique portion of the URL to the image.

I'm trying to download each image, and name of each image is based on the first 3 fields:

id1 is always 1 digit
id2 is really 5 digits with prepending zeros (so 2314 needs to become 02314)
id3 is really 4 digits (78 becomes 0078),

so in total a 10 digit code. So the first image name should be 2023140078.jpg

If I were to download a single file manually, I'd do something like this:

wget -O 2023140078.jpg http://example.com/assets/?mediafile=/image/1289.jpg

Also if I could write back some sort of status or error code to the row or write an error log, would be great.

2 Answers 2

2
awk -F, '{printf("%d%.5d%.4d %s\n",$1,$2,$3,$4)}' input.csv | while read newfile filename 
do 
wget -O "${newfile}.jpg" http://example.com/assets/?mediafile=${filename} 
done
3
  • After running this for 200,000 rows I realized that not all filenames are unique, and some IDs have several images, how can I fix this?
    – zevnyc
    Jan 17, 2017 at 20:00
  • what you mean by its not unique ? are you ok to add the incremental value in the filename ? eg: 2023140078_1.jpg 2023140078_2.jpg ..... 2023140078_200000.jpg
    – Kamaraj
    Jan 18, 2017 at 8:28
  • Some of the entries have several URLs for the same filename so wget is downloading them all, replacing the first file with the next. Most have only 1 file, but some have several. I'd like for the 1st to stay in the original format and the rest add _1 etc.
    – zevnyc
    Jan 18, 2017 at 10:25
1

You can set the IFS to a comma and use a while loop to read the 4 variables that uses the file as input:

IFS=','
while read c1 c2 c3 c4; do
  wget -O "$(printf "%d%05d%04d.jpg" "$c1" "$c2" "$c3")" "http://example.com/assets/?mediafile=${c4}"
done < file.txt
3
  • if second column has the value of 12345, then the filename will be created with 11 digit
    – Kamaraj
    Jan 17, 2017 at 6:30
  • Sorry, I misread your padding requirements. You can use printf command substitution for that: $ IFS=','; while read c1 c2 c3 c4; do echo wget -O "${c1}$(printf %05d ${c2})$(printf %04d ${c3})" "example.com/assets/?mediafile=${c4}"; done < file.txt
    – Kyle
    Jan 17, 2017 at 6:31
  • You should edit your answer to improve it.  Comments are often overlooked and sometimes deleted. And you don't need two printfs; just do $(printf "%d%05d%04d.jpg" "$c1" "$c2" "$c3"). Jan 17, 2017 at 6:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .