29

I am trying to download a file from sourceforge using wget, but as we all know we have to click on the download button and then wait for it to auto download. how do you download this type of file using wget?

I am trying to download this: http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.8.1/bitcoin-0.8.1-linux.tar.gz/download

But doing a wget on that url link would not get me the file as the file is auto loaded via the browser.

  • 3
    Going to that page in the web browser should also list a Direct Link - can you use wget with that? – BriGuy May 1 '13 at 18:58
  • If you're looking to preserve the true name of the file (rather than "download") then you merely need to invoke it as wget --trust-server-names URL – Adam Katz Jan 12 '18 at 16:45
5

I'm unsure which version of wget or OS and any proxy's exist between you and sourceforge but wget downloaded the file when I removed the "/download" and left it at the file extension.

I don't want to flood the post or pastebin my entire session but I got the 302 then 200 status codes before the transfer began. What happens when you try wget?

Resolving downloads.sourceforge.net... 216.34.181.59
Connecting to downloads.sourceforge.net|216.34.181.59|:80... connected.
HTTP request sent, awaiting response... 302 Found

[snipped for brevity]

HTTP request sent, awaiting response... 200 OK
Length: 13432789 (13M) [application/x-gzip]
Saving to: `download'
  • Also of note: occasionally the “download” page is a list of mirrors and a script which gives you one of them when a certain delay has elapsed. SourceForge 10 years ago did this. – can-ned_food Jun 4 '18 at 20:55
45

I would suggest using curl to do this instead of wget. It can follow the redirection using the switches -L, -J, and -O.

curl -O -J -L http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.8.1/bitcoin-0.8.1-linux.tar.gz/download

switch definitions

-O/--remote-name
  Write output to a local file named like the remote file we get. 
  (Only the file part of the remote  file  is  used, the path is cut off.)

-L/--location
  (HTTP/HTTPS)  If  the  server  reports that the requested page has moved 
  to a different location (indicated with a Location: header and a 3XX 
  response code), this option will make curl redo the request on the new 
  place.  If  used together  with  -i/--include  or -I/--head, headers from 
  all requested pages will be shown. When authentication is used, curl only 
  sends its credentials to the initial host. If a redirect takes curl to a 
  different host, it  won't be  able  to  intercept  the  user+password. 
  See also --location-trusted on how to change this. You can limit the
  amount of redirects to follow by using the --max-redirs option.

-J/--remote-header-name
  (HTTP) This option tells the -O/--remote-name option to  use  the  
  server-specified  Content-Disposition  filename instead of extracting a 
  filename from the URL.

See the curl man page for more details.

  • 2
    No need to use curl, wget can follow redirects, but it doesn't do so unless you increase the --max-redirect from the default, which for security reasons is of course 0. – Anthon Mar 1 '15 at 13:57
  • wget 1.19.2's default is --max-redirect=20 which should be good enough for most uses. I'm not sure when that changed … or why that was necessary for security (citation needed!), but my guess is that the current preferred solution was to follow redirections and instead require --trust-server-names to preserve the name provided by a redirection target rather than "download" or "index.html?blah=barg" or whatever the provided URL uses. – Adam Katz Jan 12 '18 at 16:49
  • In case it helps anyone, here's how I remember the correct switches. Think Jennifer Lopez. JLO. curl -JLO http://www.example.com/file.ext – Carl May 13 at 10:24
13

In wget you may use --content-disposition option which is useful for some file-downloading CGI programs that use "Content-Disposition" headers to describe what the name of a downloaded file should be.

In example:

wget --user-agent=Mozilla --content-disposition -E -c http://example.com/

For more complex solutions (like authorisation required), use cookie file (--load-cookies file) to simulate your session.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.