Questions tagged [curl]

cURL is a CLI tool for issuing HTTP GET, POST, & PUT request.

Filter by
Sorted by
Tagged with
428
votes
3answers
588k views

Does curl have a --no-check-certificate option like wget?

I am trying to make a curl request to one of our local development servers running a dev site with a self-signed SSL cert. I am using curl from the command line. I saw some blog posts mentioning that ...
245
votes
3answers
128k views

What is the difference between curl and wget?

I am keen to know the difference between curl and wget. Both are used to get files and documents but what the key difference between them. Why are there two different programs?
239
votes
5answers
375k views

Does curl have a timeout?

So far I couldn't find anything really, but is it true that curl doesn't really time out at all? user@host:~# curl http://localhost/testdir/image.jpg I'm asking because I'm redirecting any request ...
192
votes
2answers
262k views

Hide curl output

I'm making a curl request where it displays an html output in the console like this <b>Warning</b>: Cannot modify header information - headers already sent by (output started at /home/...
103
votes
2answers
151k views

Any way to encode the url in curl command?

I have some url which has space in it's query param. I want to use this in curl, e.g. curl -G "http://localhost:30001/data?zip=47401&utc_begin=2013-8-1 00:00:00&utc_end=2013-8-2 00:00:00&...
91
votes
3answers
65k views

Throttle the download speed of wget or curl while downloading

Is it possible to throttle (limit) the download speed of wget or curl ? Is it possible to change the throttle value while it is downloading ?
67
votes
2answers
133k views

How to grep the output of cURL?

I need to retrieve the expiry date of an SSL cert. The curl application does provide this information: $ curl -v https://google.com/ * Hostname was NOT found in DNS cache * Trying 212.179.180.121......
67
votes
3answers
5k views

How does curl protect a password from appearing in ps output?

I noticed some time ago that usernames and passwords given to curl as command line arguments don't appear in ps output (although of course they may appear in your bash history). They likewise don't ...
56
votes
5answers
129k views

why would curl and wget result in a 403 forbidden?

I try to download a file with wget and curl and it is rejected with a 403 error (forbidden). I can view the file using the web browser on the same machine. I try again with my browser's user agent, ...
52
votes
4answers
175k views

Set a network range in the no_proxy environment variable

I'm in a network using a proxy. I've got machines using lots of scripts here and there accessing each other over HTTP. The network is 10.0.0.0/8. My proxy is 10.1.1.1:81, so I set it up accordingly: ...
37
votes
4answers
65k views

Most straightforward way of getting a raw, unparsed HTTPS response

What would be the most straightforward way of making a GET request to a url over HTTPS, and getting the raw, unparsed response? Could this be achieved with curl? If so, what options would you need to ...
36
votes
1answer
11k views

What does the @ (at) symbol before a filename mean in a curl command?

I am looking at an example of a curl command that uses the @ symbol before the filename to post data to the server. curl http://localhost/ --data-binary @file.txt What does this mean? Does it insert ...
34
votes
2answers
32k views

Resume failed download using Linux command line tool

How do I resume a partially downloaded file using a Linux commandline tool? I downloaded a large file partially, i.e. 400 MB out of 900 MB due to power interruption, but when I start downloading ...
32
votes
8answers
150k views

Health check of web page using curl

I'd like to do a health check of a service by calling a specific url on it. Feels like the simplest solution would be to use cron to do the check every minute or so. In case of errors, cron sends me ...
32
votes
2answers
59k views

curl --resolve appears to do nothing

The --help output for curl lists a --resolve option, which states --resolve <host:port:address> Force resolve of HOST:PORT to ADDRESS I'm not having any luck getting it to work though. The ...
29
votes
7answers
10k views

Download big file over bad connection

Is there an existing tool, which can be used to download big files over a bad connection? I have to regularly download a relatively small file: 300 MB, but the slow (80-120 KBytes/sec) TCP connection ...
29
votes
2answers
49k views

Passing binary data to curl without using a @file

Is it possible to use curl and post binary data without passing in a file name? For example, you can post a form using binary via --data-binary: curl -X POST --data-binary @myfile.bin http://foo.com ...
27
votes
3answers
23k views

Why won't curl download this link when a browser will?

I am running Mac OS 10.11.6 El Capitan. There is a link I would like to download programmatically: https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.16-osx10.11-x86_64.dmg If I paste this URL ...
27
votes
3answers
21k views

How do I use curl to download content from sourceforge?

Simply typing http://downloads.sourceforge.net/project/romfs/genromfs/0.5.2/genromfs-0.5.2.tar.gz works fine on a browser, but I'm trying to download from a CLI environment with limited utilities. ...
26
votes
1answer
31k views

How do I display all URLs in a redirect chain?

I'm looking for a way to show all of the URLs in a redirect chain, preferably from the shell. I've found a way to almost do it with curl, but it only shows the first and last URL. I'd like to see all ...
26
votes
3answers
182k views

How to fix curl sslv3 alert handshake failure?

I'm trying to curl HTTPS website in the following way: $ curl -v https://thepiratebay.se/ However it fails with the error: * About to connect() to thepiratebay.se port 443 (#0) * Trying 173.245....
23
votes
6answers
6k views

What are the alternatives for checking open ports, besides telnet?

We can use the following in order to test telnet VIA port; in the following example we test port 6667: [root@kafka03 ~]# telnet kafka02 6667 Trying 103.64.35.86... Connected to kafka02. Escape ...
23
votes
5answers
46k views

Get URL's redirect target with curl

I would like to check where a single URL redirects. An example of that could be a link from Google's search result page (where a click always goes through Google server). Can I do that with curl?
23
votes
2answers
85k views

How to check whether a command such as curl completed without error?

I am using curl to upload a file to a server via an HTTP post. curl -X POST -d@myfile.txt server-URL When I manually execute this command on the command line, I get a response from the server like "...
21
votes
4answers
11k views

What's the difference between `curl | sh` and `sh -c “$(curl)”`?

One easy install method for Docker (for example) is this: curl -sSL https://get.docker.com/ | sh However, I have also seen some that look like this (using the Docker example): sh -c "$(curl -sSL ...
18
votes
2answers
62k views

Sending CURL request with custom IP

Shouldn't it be possible? Let's assume I don't need a response, I just want to send a request. Shouldn't we be able to alter tcp/ip headers, because our computer sends it? I am probably missing ...
17
votes
1answer
84k views

curl url call in for loop? [closed]

I am working with bash and I am trying to make a curl url call in a for loop. Below is my simple curl call - curl -v --header "Connection: keep-alive" "localhost:8080/user?userid=52010&client_id=...
15
votes
1answer
18k views

cUrl : Argument list too long

I'm getting the error: Argument list too long when trying to use cUrl to send a file in base64 inside the body of my JSON. I'm using something like this: DATA=$( base64 "$FILE" ) curl -X POST -H "...
15
votes
1answer
13k views

How to see list of curl ciphers?

My understanding is that during ssl negotiation, the client (i.e. curl) sends a list of ciphers to the server, and the server replies with its preferred choice. How do I see the list of ciphers that ...
14
votes
2answers
20k views

curl download multiple files with brace syntax

I am trying to download two files by the following syntax: curl -O http://domain/path/to/{file1,file2} The problem is that only the first file is actually saved locally, and the second was simply ...
13
votes
5answers
38k views

How do I get (only) the http status of a site in a shell script?

I figure curl would do the job. I wrote in a script: #!/bin/sh function test { res=`curl -I $1 | grep HTTP/1.1 | awk {'print $2'}` if [ $res -ne 200 ] then echo "Error $res on $1"...
12
votes
3answers
14k views

Automating web requests using curl?

I have some "corporative" systems I have to use at work. Some of them are terrible, with frames, pop-ups and such. I was thinking about automating some stuff using curl or something like that. I need ...
12
votes
4answers
47k views

Set timeout for web page response with curl

I use a tool to monitor if the web-page is up and running. The tool uses curl command internally to fetch the output. However, when a web-page takes longer time to respond, it results back with a ...
12
votes
2answers
27k views

Is there a command line method by which I can check whether a downloaded file is complete or broken?

I'm writing a script that involves downloading and manipulating a file, and I want to ensure that the file is not incomplete (due to e.g. a dropped connection) before I work on it.
12
votes
2answers
42k views

Running thousands of curl background processes in parallel in bash script

I am running thounsand of curl background processes in parallel in the following bash script START=$(date +%s) for i in {1..100000} do curl -s "http://some_url_here/"$i > $i.txt& ...
11
votes
1answer
20k views

Uploading multiple files via FTP using curl

I'm trying to upload all the text files within the current folder via FTP to a server location using curl. I tried the following line: curl -T "{file1.txt, file2.txt}" ftp://XXX --user YYY where ...
11
votes
3answers
5k views

What is most secure and simplest way to have a user-typed password on bash become part of stdin to a program?

I'm looking for the (1) most secure and (2) simplest way to have a user type a password on a bash shell prompt and to have that password become part of stdin to a program. This is what the stdin ...
11
votes
5answers
35k views

Write output of wget or curl to a custom filename based on the url

For example I have a link http://www.abc.com/123/def/ghi/jkl.mno. I want to download it using wget or curl and get the name of output file as def_ghi_jkl.mno, where the part def_ghi is taken from the ...
11
votes
1answer
38k views

Login site using cURL

I am trying to login from cURL command line with the command curl --data "username=user&password=pass&submit=Login" http://www.ip.com:8080/LoginApplication/Login.jsp And after that trying to ...
11
votes
1answer
2k views

how to make Gnu/Linux trust a certificate that's trusted by Windows out-of-the-box?

There is a server with a broken SSL chain, as reported by this SSL check: I know this is a problem that should be solved on the server itself, but sometimes this is hard to have fixed (I'm not the ...
11
votes
5answers
3k views

Read and confirm shell script before piping from curl to sh (curl -s [url] | sh)

Whenever I have to execute a shell script from the web curl -s [url] | sh, I first open url in my web browser to make sure the script isn't malicious and is safe to run. I remember seeing a command ...
10
votes
3answers
18k views

Merge jq output into a comma separated string [duplicate]

I am trying to curl some URL which returns a json file, then I want to parse hosts from it and create a comma separated string. I have the first part working curl -s -u "admin:admin" -H "X-...
10
votes
3answers
27k views

How to convert ssl ciphers to curl format?

The official ssl docs list ciphers in a different format than curl takes. For instance, if I want curl to use the cipher TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, I have to pass it curl --ciphers ...
9
votes
7answers
8k views

Is there a way to use curl interactively? Or is there an interactive curl/wget shell?

Imagine something like this: $ curlsh http://www.example.org > GET /foo/bar/bam ...output here... > POST /thing/pool ... ... result here.... is there a tool that lets me do that?
9
votes
2answers
22k views

How can I specify that curl (via command line) overwrites a file if it already exists?

I'm using curl in this syntax: curl -o myfile.jpg http://mysite.com/myfile.jpg If I run this command twice, I get two files: myfile.jpg myfile-1.jpg How can I tell CURL that I want it to overwrite ...
9
votes
2answers
4k views

How to pass argument to script which is input to bash

Right now I have one-liner like this: curl -fsSL http://git.io/vvZMn | bash It is downloading script and passing it to bash as stdin file. I would like to run this script with additional argument ...
9
votes
1answer
4k views

How to pipe html into lynx?

I am testing some ajax call from the console. I like get rid of the meta-information noise that is html markup and just read the content, hence I like to pipe it into a terminal browser, e.g. lynx. ...
9
votes
3answers
3k views

download only first few bytes of a source page

I am downloading the HTML source page of a website using cURL command. The problem is, I do not want most of the contents in the page. I just need the first 100 lines of the source page. Is there a ...
8
votes
1answer
2k views

Why does this curl command from firefox not download anything?

I opened https://raw.githubusercontent.com/andreafrancia/trash-cli/master/README.rst in Firefox, and copy the following curl command from Tools->Web Developer->Network: curl 'https://raw....
8
votes
6answers
4k views

How to tweet using terminal?

I would like to tweet a message using terminal. I tried something like: curl -u 'TwitterUsername':'TwitterPassword' -d status=”Your Message Here” https://twitter.com/statuses/update.xml but seems ...