1

I am trying to download dozens of PDF files located on pages linked from here:

http://machineknittingetc.com/passap.html?limit=all

Each PDF is referred to by a URL ending with /downloadable/download/sample/sample_id/[some three digit number]/.

I have tried these:

wget -r -l2 -A.pdf http://machineknittingetc.com/passap.html?limit=all
wget -r -l2 -np http://machineknittingetc.com/passap.html?limit=all -A "*.pdf"
wget -r -l2 -np http://machineknittingetc.com/passap.html?limit=all -A "*.###"

It doesn't get the PDFs.

Does it have something to do with the server not being indexed to allow me to access the URLs like a file hierarchy? Is there a way to make it work?

1

@rajaganesh87 you are guessing at the directory link numbers and are your code does not work for the actual links needed per the base link http://machineknittingetc.com/passap.html?limit=all and the (.pdf) files correlating to it.

The problem is your being blocked by the

robots.txt file

and your using the dot (.) in

    -A .pdf

Try the code below that I tested and it works.

 wget -np -nd -r -l2 -A pdf -e robots=off http://machineknittingetc.com/passap.html?limit=all 
2

Does this work for you ?

#!/bin/bash
for i in {000..175}
do
     wget  http://machineknittingetc.com/downloadable/download/sample/sample_id/$i
done
  • Yes, thanks! But it gets a lot more than the links on that page. Apparently the downloadable subpath has a lot of files. I will look for a range for the files I want (hopefully they are not randomly numbered) and see if I can alter it. – Kallaste Jan 2 '17 at 9:21
  • I really should have thought of that. – Kallaste Jan 2 '17 at 9:23
  • No, it seems the files I want are not numbered predictably. Some are consecutive but then they start to jump around. Short of passing it a list of each file path, this will not work. Is there no way to do it with the wget filters? – Kallaste Jan 2 '17 at 9:37
  • @Kallaste In that case get the html using wget and grep for the document numbers, download again from that list – rajaganesh87 Jan 4 '17 at 11:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.