Questions tagged [download]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
258
votes
3answers
138k views

What is the difference between curl and wget?

I am keen to know the difference between curl and wget. Both are used to get files and documents but what the key difference between them. Why are there two different programs?
97
votes
3answers
68k views

Throttle the download speed of wget or curl while downloading

Is it possible to throttle (limit) the download speed of wget or curl ? Is it possible to change the throttle value while it is downloading ?
59
votes
8answers
144k views

How to download a folder from google drive using terminal?

I want to download a folder from my google drive using terminal? Is there any way to do that? I tried this: $ wget "https://drive.google.com/folderview?id=0B-Zc9K0k9q-WWUlqMXAyTG40MjA&usp=sharing"...
38
votes
2answers
2k views

How do I distribute a large download over multiple computers?

I need to download a large file (1GB). I also have access to multiple computers running Linux, but each is limited to a 50kB/s download speed by an admin policy. How do I distribute downloading this ...
35
votes
3answers
70k views

How do I use wget with a list of URLs and their corresponding output files?

Suppose list_of_urls looks like this: http://www.url1.com/some.txt http://www.url2.com/video.mp4 I know how to use that with: wget -i list_of_urls But, what if my list_of_urls has this, and they ...
34
votes
2answers
34k views

Resume failed download using Linux command line tool

How do I resume a partially downloaded file using a Linux commandline tool? I downloaded a large file partially, i.e. 400 MB out of 900 MB due to power interruption, but when I start downloading ...
30
votes
7answers
12k views

Download big file over bad connection

Is there an existing tool, which can be used to download big files over a bad connection? I have to regularly download a relatively small file: 300 MB, but the slow (80-120 KBytes/sec) TCP connection ...
29
votes
5answers
50k views

How to download package not install it with apt-get command?

sudo apt-get install pppoe will download pppoe package and install it. Is it possible to just download pppoe package and not install it with apt-get command? wget http://ftp.us.debian.org/debian/...
22
votes
4answers
25k views

wget - How to download recursively and only specific mime-types/extensions (i.e. text only)

How to download a full website, but ignoring all binary files. wget has this functionality using the -r flag but it downloads everything and some websites are just too much for a low-resources ...
19
votes
2answers
22k views

Download files and create same file structure as the source

I have a config file which consists of list of URIs I want to download. For example, http://xyz.abc.com/Dir1/Dir3/sds.exe http://xyz.abc.com/Dir2/Dir4/jhjs.exe http://xyz.abc.com/Dir1/itr.exe ...
17
votes
4answers
30k views

How to download files and folders from Onedrive using wget?

How to use wget to download files from Onedrive? (and batch files and entire folders, if possible)
16
votes
7answers
17k views

command-line tool for a single download of a torrent (like wget or curl)

I'm interested in a single command that would download the contents of a torrent (and perhaps participate as a seed following the download, until I stop it). Usually, there is a torrent-client daemon ...
13
votes
1answer
2k views

Wget: convert-links and avoid redownloading already fetched files?

I am downloading data spread among multiple files that don't change once published. Therefore --timestamping is not good enough, because it is constantly checking whatever resource is changed, which ...
12
votes
12answers
35k views

Is there a way to download pure Unix?

I'm just asking out of curiosity, is there a way to obtain a 'pure' so to say copy of Unix? So, not OS X or Linux with Unix in the background, but simply Unix..
12
votes
2answers
28k views

Is there a command line method by which I can check whether a downloaded file is complete or broken?

I'm writing a script that involves downloading and manipulating a file, and I want to ensure that the file is not incomplete (due to e.g. a dropped connection) before I work on it.
12
votes
1answer
14k views

Resume an aria2 downloaded file by its *.aria2 file

I have a partially downloaded file with aria2. Next to it, is a file with the same name end with .aria2. I don't know the download link. I only have these two files. I want to know how could I resume ...
11
votes
3answers
27k views

How to use wget to download from hosting sites?

wget is a very useful tool to download stuff on the internet quickly, but can I use it to download from hosting sites, like FreakShare, IFile.it Depositfiles, Uploaded, Rapidshare? If so, how can I do ...
10
votes
2answers
12k views

Download list of files if they don't already exist

If I have a list of URLs separated by \n, are there any options I can pass to wget to download all the URLs and save them to the current directory, but only if the files don't already exist?
10
votes
2answers
405 views

Linux program to look into what you're downloading

Is there a linux program that allows you to look into your current download traffic? Something that can list all the addresses I am currently connected to and downloading from.
8
votes
3answers
44k views

How to call a service URL from bash shell script in parallel?

I have a service which I am calling from another application. Below is my service URL which I am calling - http://www.betaservice.domain.host.com/web/hasChanged?ver=0 I need to do some load test on ...
8
votes
3answers
9k views

How to check if a video is completely downloaded?

I have a bunch of videos which I want to check if they are complete or not. Some of them may be downloaded partially, but they are not faulty. How can I efficiently check if these video are completely ...
7
votes
1answer
16k views

Tar and download in one step over ssh / scp [duplicate]

(This seems like an obvious / common question, but I couldn't find it when searching - maybe I wasn't using the correct terminology...) I want to login over ssh to a remote machine, tar a directory, ...
7
votes
1answer
20k views

How to use wget or other tool to download a file, for which the link is hidden(activates on-click only)

I want to to download some subtitle files, stored in rar files form http://subs.sab.bz/ The site provides rss feeds for its new releases. Unfortunately, the link provided will open a download page, ...
7
votes
1answer
28k views

How do I download and launch a JNLP file in CentOS?

I'm using Amazon's GPU cluster to experiment with some HPC java code on CentOS. The Java application is in JNLP format and has a GUI. I've followed Amazon's getting started video, but I'm not ...
7
votes
2answers
14k views

How to move completed torrent downloads to another folder without breaking the torrent link?

I'm using Transmission on EOs and I've downloaded a bunch of torrent files which ended up my Download folder in my /home. I'd like now to move the files to my external hard drive without breaking the ...
7
votes
3answers
14k views

Download multiple URLs at once

I'm aware of wget -i as a way to download a list of URLs. The only trouble is that I need to pass some different POST data to each one, which works for single urls using wget --post-data= but not for ...
7
votes
1answer
522 views

Wget cannot validate https://ftp.gnu.org?

I'm working on Solaris and trying to upgrade Git by building Git and all of its dependencies from sources. Wget is failing to download a dependency, which is located at https://ftp.gnu.org/gnu/...
6
votes
5answers
1k views

Download Manager for Linux

I need some good download manager for Linux (like IDM on windows). I want it to be able to install a browser plugin too, which allows you to download movies from youtube or other sites easily. It ...
6
votes
2answers
6k views

Which Gentoo should I download?

I am looking for the most compatibility in a Gentoo download. When I go to the download page, I am greeted by this: install-x86-minimal-20120306.iso 07-Mar-2012 08:50 135M ...
6
votes
2answers
6k views

Download Only a Part of a File

I am trying to find if it is possible to download a part of a file (any file) via axel or aria2 or any other tools available. For example there is a movie file (because doing this kind of thing ...
5
votes
1answer
5k views

Why wget is not willing to download recursively?

The command $ wget -r http://www.comp.brad.ac.uk/research/GIP/tutorials/index.html only downloads index.html and robots.txt for me, even though there are links in it to further pages in the same ...
5
votes
2answers
4k views

Resolve filename from a remote URL without downloading a file

I am creating a script, which should download latest version of an application from repository and deploy the app. The main issue: there are several repositories and I need to check, which of them ...
5
votes
3answers
14k views

wget downloading tar.gz file as html?

I need to download this file http://download.oracle.com/otn-pub/java/jdk/7u67-b01/jdk-7u67-linux-x64.tar.gz using wget. I use the command wget http://download.oracle.com/otn-pub/java/jdk/7u67-b01/...
5
votes
1answer
3k views

How to download all html files from a URL's directory with wget?

I'd like to use wget to download a tutorial in a website, but this tutorial is in a specific directory. Suppose the URL is the tutorial main page http://www.abc.com/aaa/bbb/ccc/index.html. But the ...
5
votes
1answer
1k views

Download tool needed — with custom headers, resume, retry, custom filename output, and dynamic redirection

Simple problem -- I need a download manager, console, GUI, does not matter. The catch is in the features: custom header -- I have to pass a cookie in order to download (authorization) resume -- so if ...
5
votes
1answer
4k views

Wget: downloading files selectively and recursively?

Question about wget, subfolder, and index.html. Let's say I am inside "travels/" folder and this is in "website.com": "website.com/travels/". Folder "travels/" contains a lot of files and other (sub)...
5
votes
3answers
8k views

Download one html page with all needed graphics and linked pdf/zip files

I want to download a website from an URL, to view it locally, more exactly: Download one single html page (no other linked html pages) and everything needed to display it (css, images, etc.) Also ...
5
votes
2answers
4k views

Command-line way to download youtube video in multi-threaded

On a fast internet connection, downloading Youtube videos is too slow compared to downloading files from other websites. Currently I'm using get_flash_videos to download the Youtube videos. How can I ...
5
votes
2answers
4k views

Problem with recursive download using wget

I am trying to learn how to use recursive download with wget from the wget info page. For example, let us try to download all the images of xkcd. A list of all the pages is present in the xkcd ...
4
votes
3answers
41k views

download file from Internet to server using SSH

Recently because of some problem, I lost all my server files and requested the hosting team to provide me my files from a backup. They will provide me a link from which I have to download a compressed ...
4
votes
3answers
2k views

Is it possible to download extremely large files intelligently or in parts via SSH from Linux to Windows?

I have a ~35 GB file on a remote Linux Ubuntu server. Locally, I am running Windows XP, so I am connecting to the remote Linux server using SSH (specifically, I am using a Windows program called SSH ...
4
votes
2answers
5k views

Why is “wget -r -e robots=off http://ccachicago.org” not acting recursively?

I am trying to recursively download http://ccachicago.org, and am getting exactly one file, the root index.html, downloaded. I've looked at Download recursively with wget and started using the ...
4
votes
1answer
5k views

Continue interrupted “curl” download

I am downloading a large file over the network. while downloading (say at 40%) the underlying machine get disconnected from network and before the curl quit, the network get connected. But in this ...
4
votes
2answers
235 views

Uniting urls for a download utility (like wget) in one line

Consider these wget codes: wget -P ~/ https://raw.githubusercontent.com/user/repo/branch/papj.sh wget -P ~/ https://raw.githubusercontent.com/user/repo/branch/nixta.sh Is there any elegant way to ...
4
votes
2answers
65k views

How to install tar file (jhead) on Mac or Linux machine

I'm new to Linux and tar balls and was wondering how to properly install them on a Mac or Linux machine. I would prefer to know how to install on a mac but I just need some help understanding them. I ...
4
votes
5answers
13k views

Javascript to HTML converter?

Take for example this snippet of code on a web page. <html><body> <script language="javascript"> document.write("The cat"); document.write(" sat on the mat"); </script> </...
4
votes
2answers
1k views

Download from dynamically changing urls via script

I want to write a script to download all Walter Lewin lectures from this page in wmv format: http://videolectures.net/mit801f99_physics_classical_mechanics/ However the links to the wmv file for ...
4
votes
2answers
475 views

Linux axel write error

I was downloading a 4.0 GB file using axel. When it reached 98%, it displayed "Write Error". I still have 30+ GB of free disk space. I tried to run axel again, and got "Write Error" again. Here is ...
4
votes
2answers
2k views

Bash Script - Wget Graphical Download Progress

I'm trying to port this code from wget + zenity to wget + kdialog. The code with zenity: #!/bin/bash # Start wget | zenity # Note the & at the end of the pipe, this allows the script to continue ...
3
votes
1answer
487 views

How to tell wget where to fetch files passed via '--input-file'

I have a list of filenames (not urls) in a file, and I want to download them using --input-file option. How do I specify the url where those files will be fetched from?