Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to download images contained in some web pages, but only those that are larger than 100 KB. So far I've been downloading all the images and then deleting the small ones, but it's not very efficient. Is there some way to ignore the small ones when downloading?

share|improve this question
up vote 2 down vote accepted

If you use a http HEAD request, only the headers will be returned.

Here's a sketchy approach (assuming you have a list of urls). threshold=expr 100 \* 1024

for url in ${list_of_urls} ; do    
    size=`curl -s --head ${url} | grep 'Content-Length:' | cut -d ' ' -f 2` 

    if [ ${size} -gt ${threshold} ] ; then  
        curl -s ${url}
share|improve this answer
It is unclear, how this is relevant to the problem of downloading images. – rozcietrzewiacz Nov 18 '11 at 22:34
@rozcietrzewiacz That's apartial answer, showing how to determine which images match the length requirement and download them. The missing piece is parsing the web pages to gather the image URLs. – Gilles Nov 19 '11 at 2:19

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.