1
vote
2answers
67 views

How to get wget to use the direct URL file name as output file name for a shortened URL

I'm trying to download all the books from a URL list in bulk. The list is in a text file in http://ligman.me/W3TuLr. I'm trying to use wget on linux for this. All these URLs seem to be created using a ...
1
vote
1answer
60 views

How to burn an iso while downloading it

I'm in a real hurry so I thought there is a way to do that but I don't know how. I thought about piping wget -c cdimage.ubuntu.com/daily-live/current/utopic-desktop-amd64.iso | BurningApp but I don't ...
1
vote
0answers
84 views

Using wget to dowload images in facebook conversations

This look a bit harder than usual. I'm a new wget user and i was thinking that maybe it's possible to somehow login into facebook and download images from a specific conversation. I mean, i'd like ...
2
votes
3answers
859 views

How to download a folder from google drive using terminal?

I want to download a folder from my google drive using terminal? Is there any way to do that? I tried this: ‍ wget ...
4
votes
2answers
232 views

Is there a command line method by which I can check whether a downloaded file is complete or broken?

I'm writing a script that involves downloading and manipulating a file, and I want to ensure that the file is not incomplete (due to e.g. a dropped connection) before I work on it.
1
vote
2answers
162 views

Resolve filename from a remote URL without downloading a file

I am creating a script, which should download latest version of an application from repository and deploy the app. The main issue: there are several repositories and I need to check, which of them ...
2
votes
3answers
892 views

Download multiple URLs at once

I'm aware of wget -i as a way to download a list of URLs. The only trouble is that I need to pass some different POST data to each one, which works for single urls using wget --post-data= but not for ...
1
vote
2answers
120 views

Why is “wget -r -e robots=off http://ccachicago.org” not acting recursively?

I am trying to recursively download http://ccachicago.org, and am getting exactly one file, the root index.html, downloaded. I've looked at Download recursively with wget and started using the ...
3
votes
1answer
543 views

Problem with recursive download using wget

I am trying to learn how to use recursive download with wget from the wget info page. For example, let us try to download all the images of xkcd. A list of all the pages is present in the xkcd ...
0
votes
2answers
109 views

download multiple files from sourceforge

Is there any way to download / mirror a folder structure with many files from sourceforge using wget/curl/similar tool? I've seen some examples using rsync or scp but I think that works only for SF ...
0
votes
2answers
57 views

Not able to download the content of the web site

I'm using Ubuntu 13.04. I opened this website in my browser. The site starts downloading and displaying previews of a book on the same page. I'd like to download the book displayed on the page to my ...
1
vote
3answers
305 views

Download one html page with all needed graphics and linked pdf/zip files

I want to download a website from an URL, to view it locally, more exactly: Download one single html page (no other linked html pages) and everything needed to display it (css, images, etc.) Also ...
2
votes
2answers
656 views

wget grabbing empty files that actually exist

I have a script that's meant to download a certain number of files from a remote server. It only needs to do this every 24 hours, since they are JSON sources for a database on my server. The files are ...
3
votes
1answer
722 views

How to download all html files from a URL's directory with wget?

I'd like to use wget to download a tutorial in a website, but this tutorial is in a specific directory. Suppose the URL is the tutorial main page http://www.abc.com/aaa/bbb/ccc/index.html. But the ...
2
votes
1answer
434 views

Why is wget skipping files while trying to mirror a site

I ran wget on this site: http://demo.codestag.com/geeklove/ but it only returns a single index.html wget --recursive --no-clobber --page-requisites --html-extension --convert-links ...
5
votes
3answers
4k views

wget - How to download recursively and only specific mime-types/extensions (i.e. text only)

How to download a full website, but ignoring all binary files. wget has this functionality using the -r flag but it downloads everything and some websites are just too much for a low-resources ...
4
votes
2answers
383 views

Download from dynamically changing urls via script

I want to write a script to download all Walter Lewin lectures from this page in wmv format: http://videolectures.net/mit801f99_physics_classical_mechanics/ However the links to the wmv file for ...
37
votes
3answers
8k views

What is the difference between curl and wget?

I am keen to know the difference between curl and wget. Both are used to get files and documents but what the key difference between them. Why are there two different programs?
5
votes
1answer
484 views

Download tool needed — with custom headers, resume, retry, custom filename output, and dynamic redirection

Simple problem -- I need a download manager, console, GUI, does not matter. The catch is in the features: custom header -- I have to pass a cookie in order to download (authorization) resume -- so ...
4
votes
1answer
5k views

How to use wget to download from hosting sites?

wget is a very useful tool to download stuff on the internet quickly, but can I use it to download from hosting sites, like FreakShare, IFile.it Depositfiles, Uploaded, Rapidshare? If so, how can I do ...
16
votes
4answers
8k views

Throttle the download speed of wget or curl while downloading

Is it possible to throttle (limit) the download speed of wget or curl ? Is it possible to change the throttle value while it is downloading ?
4
votes
2answers
2k views

Download list of files if they don't already exist

If I have a list of URLs separated by \n, are there any options I can pass to wget to download all the URLs and save them to the current directory, but only if the files don't already exist?
0
votes
1answer
3k views

How to use wget or other tool to download a file, for which the link is hidden(activates on-click only)

I want to to download some subtitle files, stored in rar files form http://subs.sab.bz/ The site provides rss feeds for its new releases. Unfortunately, the link provided will open a download page, ...
1
vote
1answer
139 views

Alert when a process exits

I often download files using wget and cURL. Is there anyway I can make my Linux box give a visual alert when either of those processes exit (i.e: the download completes) PS : I am using Ubuntu ...
2
votes
1answer
578 views

How to wget website with same page and content names?

I'm trying mirror a website where it has page: http://www.site.com/news and contents: http://www.site.com/news/36-news/news-one http://www.site.com/news/37-news/news-two ...
3
votes
1answer
332 views

How to tell wget where to fetch files passed via '--input-file'

I have a list of filenames (not urls) in a file, and I want to download them using --input-file option. How do I specify the url where those files will be fetched from?