334

The main differences are: wget's major strong side compared to curl is its ability to download recursively. wget is command line only. There's no lib or anything, but curl's features are powered by libcurl. curl supports FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS, FILE, POP3, IMAP, SMTP, RTMP and RTSP. wget supports HTTP, HTTPS and ...


82

They were made for different purposes wget is a tool to download files from servers curl is a tool that let's you exchange requests/responses with a server wget Wget solely lets you download files from an HTTP/HTTPS or FTP server. You give it a link and it automatically downloads the file where the link points to. It builds the request automatically. ...


60

The common protocols HTTP, FTP and SFTP support range requests, so you can request part of a file. Note that this also requires server support, so it might or might not work in practice. You can use curl and the -r or --range option to specify the range and eventually just catting the files together. Example: curl -r 0-104857600 -o distro1.iso '...


58

Since you didn't specify, I'm assuming you are using wget to download the file. If this is the case, try using it with the -c option (e.g. wget -c <URL>). Please notice that in case the protocol used is ftp (the URL looks like ftp://...) there is a chance the remote server uses an old/ancient ftp daemon which doesn't support resuming downloads (newer ...


55

Use --download-only: sudo apt-get install --download-only pppoe This will download pppoe and any dependencies you need, and place them in /var/cache/apt/archives. That way a subsequent apt-get install pppoe will be able to complete without any extra downloads.


50

I was able to download a public shared file using this command: $ wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=FILEID' -O FILENAME Where FILEID must be replaced by the actual file ID. FILENAME is the path/filename where download will be stored. Note you cannot use a folderid instead of fileid. I have used view source in ...


37

By default, wget writes to a file whose name is the last component of the URL that you pass to it. Many servers redirect URLs like http://www.url1.com/app?q=123&gibb=erish&gar=ble to a different URL with a nice-looking file name like http://download.url1.com/files/something.pdf. You can tell wget to use the name from the redirected URL (i.e. ...


35

lftp (Wikipedia) is good for that. It supports a number of protocols, can download files using several concurrent parallel connections (useful where there's a lot of packet loss not caused by congestion), and can automatically resume downloads. It's also scriptable. Here including the fine-tuning you came up with (credits to you): lftp -c 'set net:idle 10 ...


33

From man wget: -x, --force-directories: [...] create a hierarchy of directories, even if one would not have been created otherwise. E.g. wget -x http://fly.srk.fer.hr/robots.txt will save the downloaded file to fly.srk.fer.hr/robots.txt.


30

Use this command: sudo apt-get download pppoe this command let you download the package and requirements into the current directory. Also for accessing to installed .deb files, you can check this path: /var/cache/apt/archives


22

You could specify a list of allowed resp. disallowed filename patterns: Allowed: -A LIST --accept LIST Disallowed: -R LIST --reject LIST LIST is comma-separated list of filename patterns/extensions. You can use the following reserved characters to specify patterns: * ? [ ] Examples: only download PNG files: -A png don't download CSS files: -R css ...


22

To get the structure you are asking for, I would suggest using -nH as well as -x. This will remove the hostname as well as create the expected directory structure. e.g. wget -x -nH http://xyz.abc.com/Dir1/Dir3/sds.exe - 'Dir1/Dir3/sds.exe' saved [1234] From the man page: -nH --no-host-directories Disable generation of host-prefixed directories. By ...


20

Actually, the major difference is that curl includes a library (libcurl), and that library is widely used by other applications. wget is standalone.


19

There is one way that works for me (based on How to Make Direct Link of OneDrive Files) Right-click on the file you are interested in download (from web interface), and choose Embed. Press "Generate HTML code to embed this file". Copy the part contained in the "" of src is your link. This will look like . Replace embed with download. This will look like ...


17

I tested this, and found the issue: wget respects robots.txt unless explicitly told not to. wget -r http://www.comp.brad.ac.uk/research/GIP/tutorials/index.html --2015-12-31 12:29:52-- http://www.comp.brad.ac.uk/research/GIP/tutorials/index.html Resolving www.comp.brad.ac.uk (www.comp.brad.ac.uk)... 143.53.133.30 Connecting to www.comp.brad.ac.uk (www.comp....


16

If you follow your description the command becomes "obvious" (at least, when you see it...). You really weren't far off with your attempt, though: ssh user@remote tar czf - /my/directory/ > /my/local/destination/archive.tgz (The "duplicate" link cited by Marco provides a cleaner solution. To create this command here I've simply followed the style of ...


15

You may use the wget utility. It has a really simple syntax, and all what do you need is to: wget http://link.to.file and it will be stored in the same directory where do you run wget. If you'd like to store a downloaded file somewhere else, you may use -P option, e.g. wget -P /path/to/store http://link.to.file


14

Launch it with javaws (java webstart), e.g javaws ~/Downloads/ContestAppletProd.jnlp


13

I wouldn't call it multithreading as such but you could simply launch 70 jobs in the background: for i in {1..70}; do wget http://www.betaservice.domain.host.com/web/hasChanged?ver=0 2>/dev/null & done That will result in 70 wget processes running at once. You can also do something more sophisticated like this little script: #!/usr/bin/env bash ...


13

I can't test this for you in your situation, but you should not be using --range with -C -. Here's what the man page has to say on the subject: Use -C - to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out. Try this instead: curl -s --retry 9999 --retry-delay 3 --speed-...


11

This kind of site tries to make it difficult not to use a graphical web browser, because if you use wget you'll be missing all these ads that pay for the bandwidth. Some sites don't make advanced checks and can be tricked easily: tell wget to pretend that it's really Mozilla and that it's coming from the download site. wget --user-agent='Mozilla/5.0 (...


11

A newer, open-source multi-platform client, written in Go called drive is quite nice and full-featured, and also is in active development. Examples The pull command downloads data from Google Drive that does not exist locally, and deletes local data that is not present on Google Drive. Run it without any arguments to pull all of the files from the ...


11

Check out transmission-cli. The usage is as simple as running transmission-cli <torrent-file>, but you can obviously tune it to your needs with several options. Just a side comment: Actually you could use many other options, apart from transmission-cli and there will probably appear many other suggestions here (like deluge by Benjamin B. in the ...


11

you can use curl -L -O --retry 999 --retry-max-time 0 -C - http://url --retry 999 : retrying so many times --retry-max-time 0 : prevent it from timing out on retrying or curl -L -o 'filename' -C - http://url Update export ec=18; while [ $ec -eq 18 ]; do /usr/bin/curl -O -C - "http://www.example.com/big-archive.zip"; export ec=$?; done ...


10

check iftop and nload iftop does for network usage what top(1) does for CPU usage. It listens to network traffic on a named interface and displays a table of current bandwidth usage by pairs of hosts. Handy for answering the question "why is our ADSL link so slow?". nload is a console application which monitors network traffic and bandwidth usage in ...


10

I did some performance tests with wget and curl, and the result is: 100 times tested average run time while download 1MB file: wget: 0.844s cURL: 0.680s 100 times tested average run time while download 5MB file: wget: 1.075s cURL: 0.863s 100 times tested average run time while download 10MB file: wget: 1.182s cURL: 1.074s Command size on the system: ...


10

The most common way to verify the integrity of downloaded files is to use MD5 checksums. This assumes that the site you are downloading from actually published MD5 checksums of their files. You can verify a MD5 checksum by creating your own checksum of the downloaded file and comparing it to the published checksum. If they are identical the file you have ...


10

There is a giant program in the ubuntu 17.04 repository called: rclone $ sudo apt update && sudo apt install rclone $ rclone config --> than follow the steps to setup your gdrive now you can rclone copy and sync everything you want. The projects supports all kinds of clouds.. f.e.: $ rclone sync /home/<usrname>/Desktop/yourfolder ...


10

Depending on the download type, you should use either or both of the following two options: aria2c --continue=true \ --input-file=${path_to_./aria2_script} But, you will apparently need the url for the download with both options. As far as I can tell, you cannot resume a download without it. I'm sorry this couldn't be better help, but the docs are here. ...


10

The easiest way I found was to select all torrents in Transmission, then go to the menu Torrent > Set Location and then choose the desired location for torrents. After which, Transmission takes care of moving torrents.


Only top voted, non community-wiki answers of a minimum length are eligible