1

This question already has an answer here:

I was asking myself how to download

  • all (.svg) images
  • from a single wikipedia article

without doing it manually for each single image. Wikipedia is offering images in different resolutions and several formats using a more complex looking url-scheme

Example url:

https://en.wikipedia.org/wiki/Thirty-six_Views_of_Mount_Fuji

marked as duplicate by G-Man, Romeo Ninov, sebasth, taliezin, Thomas Sep 12 '18 at 10:06

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • Regarding duplicate: my question and the related answer is wikipedia specific and contains informations how to change the default url to get the full-resolution images. – dufte Sep 12 '18 at 10:58
  • Closers: a subset is not a duplicate of a superset. – agc Sep 12 '18 at 11:28
2

I found a perfect solution at dougie.io's Using Wget, Grep, and Sed to Download Public Domain Wallpapers From a Web Page. I'm trying to boil it down to the main steps (to keep it available here as well):

  1. Download the HTML page using wget:

    wget https://en.wikipedia.org/wiki/Thirty-six_Views_of_Mount_Fuji \
         -O page.html -O page.html
    
  2. Extract the image URLs using grep and sed. Then extract the URLs from the article and write them to a new file urls.txt:

    grep -E "(https?:)?//[^/\s]+/\S+\.(jpg|png|gif|svg)" page.html -o | 
    sed "s/(^https?)?\/\//https\:\/\//g" -r > urls.txt
    
  3. Download the images using wget:

    • Thumb images
      If you just need the thumb images you can start that via:

      wget -i urls.txt -P downloads/
      
    • Full size images
      To get the full size images, filter the URLs file (urls.txt) to a new file (urls-new.txt):

      sed -E "s/\/thumb//g; s/\/[0-9]+px-.+\.(jpg|png)$//g" urls.txt |
      uniq > urls-new.txt
      

      then restart the download:

      wget -i urls-new.txt -P downloads_full_size/
      

Full credit goes to the linked article.

Not the answer you're looking for? Browse other questions tagged or ask your own question.