I use wget to download all the html pages in an online gallery - in order to grab all the links to the full size images as follows:

wget --wait=3 --user-agent=Webtography --no-cookies --timestamping --recursive --level=2 --convert-links --no-parent --page-requisites --adjust-extension --max-redirect=0 --exclude-directories=blog http://example.com/jasper1123/a434784.html

It successfully downloads all the pages in the user's specific gallery and doesn't crawl anywhere else - good.

It doesn't grab any images.

I use wget to do this to different galleries and it works well - took some tinkering to get wget to grab everything and stay within the gallery - that being example.com/jasper1123.

After downloading I use cat, sed, and awk commands to create a text file containing only links to the images.

I plan on importing these image links into bulk image downloader and in order grab them and keep the downloads sorted.

Here is the problem I have encountered with this website:

The links to the full size images are not standard html - they are in a script as follows:


var e='.jpg',t='b',i='14712583',h='0.us.is.example.com',s='/',n='WIV',u='jasper1123/3/example.com_'+i+n.charAt(2)+n.charAt(0)+n.charAt(1); document.getElementById('big_pic').src='http://'+t+h+s+u.charAt(0)+s+u+e;




Is there a way to convert the LINK SCRIPTs to ACTUAL LINKs?

I'd like to do this in a batch process - terminal command would be great - all the LINK SCRIPTS are in a single text file.

The only solution that is coming to mind right now is importing the text file into Libre Calc and converting it using a formula.

Any thoughts?

Big thanks.

  • See wget + JavaScript? – Thomas Dickey Jun 12 '16 at 22:09
  • If you are willing to / the "script for image" part is easy to obtain - you could use a JavaScript shell. For example with nodejs as a start: js -e "var e='.jpg',t='b',i='14712583',h='0.us.is.example.com',s='/',n='WIV',u='jasper1123/3/example.com_'+i+n.charAt(2)+n.charAt(0)+n.charAt(1); console.log('http://'+t+h+s+u.charAt(0)+s+u+e);" – Runium Jun 13 '16 at 0:02
  • JS shells at MDN: developer.mozilla.org/en-US/docs/Web/JavaScript/Shells – Runium Jun 13 '16 at 0:04
  • Thanks - it worked great - I was investigating using a javascript shell and was reading up on one while you posted - I will do some experiments. I can output the links to a text file by adding > normal_links.txt to the end of the command as output. – speld_rwong Jun 13 '16 at 0:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.