I would like to download PDF files for particular search terms from the CIA archives. For example for "Vietnam", the URL is https://www.cia.gov/library/readingroom/search/site/Vietnam.

The PDF files can viewed after clicking each link, is it possible to download the PDF files programatically by modifying wget -A pdf https://www.cia.gov/library/readingroom/search/site/Vietnam

Thank you!


Maybe this can help!

First, store all the download files or URLs in a text file as:

$ cat > download-file-list.txt





echo "Enter text"


cat URLs.txt | grep -i $text > /tmp/new.txt

wget -i /tmp/new.txt

  • Thank you very much for your reply. The issue is there are 2422 pages of search results for Vietnam with 48,423 links. The PDF for each link can be viewed after clicking each link. e.g. for the first link, the URL is link – miluji Jan 23 '17 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.