curl for website without intermediate certificate

Question

I need to make request to website using curl, but website has does not provide intermediate certificate in tls.

What I mean:

When connecting to server using TLS, it MUST (rfc5246 F.1.1, rfc5246 7.4.2) give to client all certificates that are enough to validate server certificate. This means that when connecting to www.google.com, it gives me host certificate (*.google.com) and intermediate (GTS CA 1C3) and root (GTS Root R1). And when connecting to problem site, it gives only host certificate.

Example: openssl s_client -connect google.com:443 -showcerts 2>&1 < /dev/null - gives you 3 certificates.

For most browsers it's OK but not for CURL.

So I have to use curl --insecure or to update manually bundle for this site.

I'd like to make my connection secure, but automatically.

Is there some kind of tool that can build a full certificate chain from url, validate it against Root CA bundle and output value for using with curl --insecure --pinnedpubkey <value> or nothing on validation failure ?

Answer 1

curl handling of this case depends on the build.

curl can use any of several TLS implementations, determined at compile time, and it should not be much of a surprise that the behavior of the TLS implementation in curl depends on the TLS implementation in curl.

curl with schannel CAN fetch a missing chain cert (not sent by the server, as required by RFCs) using AIA, and validate the result. In addition, curl with schannel will fill-in a missing chain cert if it is in the Windows cert store (just as it will always validate against roots in the Windows cert store), and while I can't easily verify exactly when such chain certs get put in the Windows store, my (Win 10 Home) system, used daily for good bit of web browsing and occasionally for things like curl and powershell iwr, does have about a dozen of them. curl with schannel is only available on Windows, of course. And since you suggest you are using --pinnedpubkey which is not supported with schannel, you may not be using schannel.

OpenSSL definitely does not ever use AIA for this, so curl with OpenSSL, popular on many (but not all) Linux distros, does not either. As Frederic discovered, OpenSSL will fill-in a chain cert (as well as a root) from the file supplied as --cacert, so if you either know or can guess what chain cert(s) you will need, and download them (all) in advance, this will work.

I'm pretty sure NSS itself can do AIA (because Firefox does), but I don't know if and how curl would access or enable it, and the one curl with NSS I can easily test (on Ubuntu 16.04 on WSL, admittedly a bit out of date) does not do it..

I don't know about GnuTLS either way, but curl with GnuTLS on CentOS 7 does not do it.

However, to your real question, I don't of any tool that will do it manually and save the result.

https://www.ssllabs.com/ssltest/ does a very extensive test of SSL/TLS features and compatibility which includes filling-in and validating your cert/chain in a browser-like fashion and displaying the results; if will even display multiple trust paths if it finds them, which browsers don't do. You might be able to scrape those results. https://whatsmychaincert.com/ does only the chain build/correction, and I seem to recall it produces a nice PEM, but I can't verify since the only public test I know of is currently broken (the same one Frederic referenced, incomplete-chain.badssl.com).

Java (more exactly JSSE, the SSL/TLS implementation in Java when using the 'standard' cryptoproviders) can fetch using AIA as part of validation (but not by default, you must enable it) so you could write a simple Java program that connects to a website, validates the cert with AIA.caIssuers enabled, and if successful write out the resulting cert and/or its key for pinning; I'd expect this is only 20-30 lines and if no better solution is provided and I have time later I'll give it a shot. (Or, this being Stack, anyone else can do so.)

Answer 2

curl can handle self-signed certificates.

You just have to type curl --cacert fullchain.pem URL where fullchain.pem is the certificate you have downloaded trought a trusted path.

If you don't feed curl with an appropriate root, curl can't trust the server and you will need the --insecure option to make it happy.

Note that with https://regery.com/en/security/ssl-tools/self-signed-certificate-generator (or other tools), anybody can make a self signed certificate for any server. Then curl have no way to trust a server with a self-signed certificate if you don't give this certificate with an other way.

With other certificates, it is easier, because curl can trust casual root certificates (Verisign, DigiCert, LetsEncrypt... see /etc/ssl/certs/), then curl has a root certificate which is linked to the server certificate. With a server certificate, you have to propose yet an other root.