How to download all images whose size>100kb in a webpage?

Question

I want to download images contained in some web pages, but only those that are larger than 100 KB. So far I've been downloading all the images and then deleting the small ones, but it's not very efficient. Is there some way to ignore the small ones when downloading?

Answer 1

If you use a http HEAD request, only the headers will be returned.

Here's a sketchy approach (assuming you have a list of urls). threshold=expr 100 \* 1024

for url in ${list_of_urls} ; do    
    size=`curl -s --head ${url} | grep 'Content-Length:' | cut -d ' ' -f 2` 

    if [ ${size} -gt ${threshold} ] ; then  
        curl -s ${url}
    fi    
done