wget is a very useful tool to download stuff on the internet quickly, but can I use it to download from hosting sites, like FreakShare, IFile.it Depositfiles, Uploaded, Rapidshare? If so, how can I do that?
This kind of site tries to make it difficult not to use a graphical web browser, because if you use wget you'll be missing all these ads that pay for the bandwidth.
wget --user-agent='Mozilla/5.0 (Windows NT 6.0) Gecko/20100101 Firefox/14.0.1' \ --referer=http://downloadsite.example.com/download-page-url http://downloadsite.example.com/download-page-url/filename.ext`
Most sites that check let you get away with
--referer set to the URL of the file you're downloading.
With some sites, you might need to export the web browser cookies and pass
--load-cookies to wget; at this point using wget starts to be more work than a manual download. The ultimate method, which can work all the time but requires some coding for each site, is to use a browser automation framework such as Selenium or Watir.
See also Automating web requests using curl?
This method builds up on Gilles' answer. Like he said, the trick is to make the server think that the request is being made from a browser and to make that happen, you need a couple of things:
- The Download Link (The actual link to the file)
- Link Referrer (The webpage with the download button)
- Zippyshare Session ID (Found in Cookies)
Here's a screenshot explaining where you can get each item:
Now open up your terminal, and use the following command (replacing necessary items):
wget <download_link> \ --referer='<referrer>' \ --cookies=off --header "Cookie: JSESSIONID=<session_id>" \ --user-agent='Mozilla/5.0 (Windows NT 6.0) Gecko/20100101 Firefox/14.0.1'
wget http://www16.zippyshare.com/d/29887835/8895183/hello.txt \ --referer='http://www16.zippyshare.com/v/29887835/file.html' \ --cookies=off --header "Cookie: JSESSIONID=26458C0893BF69F88EB5743D74FE0F8C" \ --user-agent='Mozilla/5.0 (Windows NT 6.0) Gecko/20100101 Firefox/14.0.1'
Note: In the command, it is in fact
I also would like to above perfect answers, I used following trick for downloading a child movie for my son from Turbobit:
Instead of SESSION_ID from above answer, you need to find "compid" from the cookie tab of FIREBUG:
After that, the command for my debian embedded system was soo easy:
wget --referer='http://http://turbobit.net/download/free/yilmacr4e351' --cookies=off --header "Cookie: compid=c9255510c34624435292d8a0e7e405be66aa01f9%7ED01343F87AB53A2C38D957A564DA1D55" --user-agent='Mozilla/5.0 (Windows NT 6.0) Gecko/20100101 Firefox/14.0.1' http://turbobit.net/download/redirect/B3EDACDA9B899937A149D5AAB6662327/yilmacr4e351/Ters.Yuz.2015.1080P.WEBDL.TR.EN.mkv &
I can shut down my computer now :)