Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
I am trying to download a file from sourceforge using wget, but as we all know we have to click on the download button and then wait for it to auto download. how do you download this type of file using wget?
I am trying to download this: http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.8.1/bitcoin-0.8.1-linux.tar.gz/download
But doing a wget on that url link would not get me the file as the file is auto loaded via the browser.
I would suggest using curl to do this instead of wget. It can follow the redirection using the switches -L, -J, and -O.
curl -O -J -L http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.8.1/bitcoin-0.8.1-linux.tar.gz/download
-O/--remote-name
Write output to a local file named like the remote file we get.
(Only the file part of the remote file is used, the path is cut off.)
-L/--location
(HTTP/HTTPS) If the server reports that the requested page has moved
to a different location (indicated with a Location: header and a 3XX
response code), this option will make curl redo the request on the new
place. If used together with -i/--include or -I/--head, headers from
all requested pages will be shown. When authentication is used, curl only
sends its credentials to the initial host. If a redirect takes curl to a
different host, it won't be able to intercept the user+password.
See also --location-trusted on how to change this. You can limit the
amount of redirects to follow by using the --max-redirs option.
-J/--remote-header-name
(HTTP) This option tells the -O/--remote-name option to use the
server-specified Content-Disposition filename instead of extracting a
filename from the URL.
See the curl man page for more details.
wget can follow redirects, but it doesn't do so unless you increase the --max-redirect from the default, which for security reasons is of course 0.
wget 1.19.2's default is --max-redirect=20 which should be good enough for most uses. I'm not sure when that changed … or why that was necessary for security (citation needed!), but my guess is that the current preferred solution was to follow redirections and instead require --trust-server-names to preserve the name provided by a redirection target rather than "download" or "index.html?blah=barg" or whatever the provided URL uses.
Jan 12, 2018 at 16:49
curl -JLO http://www.example.com/file.ext
In wget you may use --content-disposition option which is useful for some file-downloading CGI programs that use "Content-Disposition" headers to describe what the name of a downloaded file should be.
In example:
wget --user-agent=Mozilla --content-disposition -E -c http://example.com/
For more complex solutions (like authorisation required), use cookie file (--load-cookies file) to simulate your session.
I'm unsure which version of wget or OS and any proxy's exist between you and sourceforge but wget downloaded the file when I removed the "/download" and left it at the file extension.
I don't want to flood the post or pastebin my entire session but I got the 302 then 200 status codes before the transfer began. What happens when you try wget?
Resolving downloads.sourceforge.net... 216.34.181.59
Connecting to downloads.sourceforge.net|216.34.181.59|:80... connected.
HTTP request sent, awaiting response... 302 Found
[snipped for brevity]
HTTP request sent, awaiting response... 200 OK
Length: 13432789 (13M) [application/x-gzip]
Saving to: `download'
wget --trust-server-names URL