I want to download all the documents pointed to by a web page with wget.

When I run wget, it only downloads the webpage, not all the contents that the web page links to.

wget --content-disposition -v -S -r -nc -np -Kk -px --ignore-length \
     --content-disposition --proxy-user "XYZ" --proxy-passwd "PASS" \
     --no-check-certificate --load-cookies=cookies.txt --keep-session-cookies \

Wget downloads only one page which is https://www.website.com/link/live?func=lab&objId=174914395&objAction=browse&viewType=1. That webpage contains:

DataStringToVariables( '( {"currentPageNum":1,"totalPages":1,"totalCount":21,"dbTimeTaken":0,"timeTaken":1,"haveBigImages":"fa lse1","myrows":[{"dataId":"177020845","type2":"184","typeName":"Document","name":"Action_Mar_JKill%2Exls", etc.

There are many xls,docs, pdf, etc but wget doesn't download them. How can I download them?

1 Answer 1


Not sure if I'm getting your question, but I'm going to try and answer anyway.

What you're getting from the URL is JSONP, but JSONP isn't handled by wget at all. wget is just a downloading program, as noted here in the man page for wget:

GNU Wget is a free utility for non-interactive download of files from the Web.

wget does not support any parsing of the data it receives. If you wish to parse the data, use another programming language to import and process the data received here.

Assuming that you're writing some sort of shell script, you may also wish to use jsawk to fetch the data. See this answer for an example.

Once you have parsed the JSON(P), you can go ahead and download the files listed from the JSON(P) data with wget.

  • When passed the -r flag, wget can parse HTML data for links.
    – user26112
    Commented Aug 14, 2013 at 23:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .