Can wget download reveal.js image assets?

Question

In using the wget command below,

$ wget \
     --recursive \
     --no-clobber \
     --page-requisites \
     --html-extension \
     --convert-links \
     --restrict-file-names=windows \
     --domains grantmlong.com \
     --no-parent \
         grantmlong.com/teaching/index.html

I have been trying to download all content from a professor's course page. For some reason, while much of the image content for the remainder of the site is being downloaded correctly, the images for the reveal.js lecture slides are not being downloaded.

For example, if, on my local computer, I navigate to grantmlong.com/teaching/lectures/ and open lecture1.html, what appears for the third slide is

Instead of this:

On the website, I find that the image is located at https://grantmlong.com/teaching/lectures/img/hbr.png. If I navigate to the local img folder downloaded by wget, I see

cd grantmlong.com/teaching/lectures/img
ls -1
l10_f0.png
l10_f1.png
l10_f2.png
l10_f3.png
l10_f4.png
l10_f5b.png
l10_f5.png
l10_f6.png
l10_f7.png
l10_p1.png
l10_p2.png
l11_p1.png
l11_p2.png
l11_p3.png
l11_p4.png
l11_p5.png
l11_p6.png
l12_p1.png
l12_p2.png
l5_e1.png
l5_e2.png
l5_e3.png
l5_e4.png
l5_glm.png
l5_logreg.png
l5_p10.png
l5_p11.png
l5_p1a.png
l5_p1b.png
l5_p2.png
l5_p3.png
l5_p4.png
l5_p5.png
l5_p6.png
l5_p7.png
l5_p8.png
l5_p9.png
l5_reg_output_1.png
l5_reg_output_2.png
l5_reg_output_3.png
l5_reg_output_4.png
l5_reg_output.png
l6_accuracy.png
l6_confusion.png
l6_p1.png
l6_precision.png
l6_recall.png
l9_p1.png
l9_p2.png
l9_p3.png
l9_p4.png
l9_t1.png
l9_t2.png
l9_t3.png
l9_t4.png
l9_t5.png

hbr.png is nowhere to be found, which shows that the images in these reveal.js slides are not considered "page requisites" and are not being downloaded by wget.

What can I do to ensure these images are downloaded? Also, note that some of the images on the reveal.js slides come from 3rd party sites like giphy. How can I ensure that this external content is downloaded while keeping the option --domains grantmlong.com true for all pages that aren't reveal.js slides?

Answer 1

No it can't.

The file hbr.png is referenced by a custom attribute data-src on the img tag.

As explained here, there is no way for wget to follow custom attributes.

Answer 2

After some more searching, I found a (hacky) solution to the problem of downloading an archive of reveal.js slides. On the codimd github, the user "zeigerpuppy" posted the following response:

I have found a way to save an archive of a slide presentation built with codimd. I had some trouble getting wget to pull the images from the presentation (I think because the links to the images are in markdown).

So, it's a three step process but it's quick and works well. Let's say you have a slide show at https://codimd.server.net/p/S1PIjfhM8#/

1. use wget to grab the files and the requisites (.css and .js)

   • your presentation will end up as p/S1PIjfhM8.html

`

wget   --recursive   --no-clobber   --page-requisites   \
--html-extension   --convert-links   \
--domains codimd.server.net \
https://codimd.server.net/p/S1PIjfhM8#/

1. use the firefox plugin: Image Picka

   • use the save pattern: Image_Picka/uploads/${name}${ext}
   • it gets all images on page (including .svg)
   • move the images to the folder called uploads in the web archive root

2. we need to use sed to change the links in the html file to relative links so that they point to the images

`

cd p
sed -i .bak 's|/uploads/upload_|../uploads/upload_|g' S1PIjfhM8.html

Then you'll have a full copy of the slides that you can run offline. It's also good for archive purposes.

It'd be great if something like this was also built into the codimd program under the save options, maybe save slides.

I took a similar approach, although I didn't do the last step with sed. Instead, I used Image Picka to download all the images missed by wget and I put them in the grantmlong.com/teaching/lectures/img/ directory on my local wget archive. That made most of the image content appear in the slides.

Although the gifs from 3rd party sites won't load, those were mostly aesthetic (no important equations or diagrams were in .gif format.) So, I'm happy that I can view the most essential content offline.