2

I need to make request to website using curl, but website has does not provide intermediate certificate in tls.

What I mean:

When connecting to server using TLS, it MUST (rfc5246 F.1.1, rfc5246 7.4.2) give to client all certificates that are enough to validate server certificate. This means that when connecting to www.google.com, it gives me host certificate (*.google.com) and intermediate (GTS CA 1C3) and root (GTS Root R1). And when connecting to problem site, it gives only host certificate.

Example: openssl s_client -connect google.com:443 -showcerts 2>&1 < /dev/null - gives you 3 certificates.

For most browsers it's OK but not for CURL.

So I have to use curl --insecure or to update manually bundle for this site.

I'd like to make my connection secure, but automatically.

Is there some kind of tool that can build a full certificate chain from url, validate it against Root CA bundle and output value for using with curl --insecure --pinnedpubkey <value> or nothing on validation failure ?

7
  • Could you explain "does not provide intermediate certificate" ? A certificate chain typically contains a root certificate you should trust, eventually intermediate certificates (this is not mandatory), and the server certificate. When all of this is consistent, it should make curl happy. What can be the root of your problem is a self-signed certificate, which - by default - you don't trust (in this case there are no intermediate certificate, but this is not the root of the issue). With such certificates, you have to configure curl to trust this certificate. Commented Mar 21, 2023 at 20:22
  • @FrédéricLoyer server MUST ( rfc5246 F.1.1, rfc5246 7.4.2) provide all certificates in chain (means host certificate and all intermediates if they exist in chain at least). But in my case, it provides only host certificate. This makes curl unhappy, as there is no known intermediate certificate between curl's Root CA bundle and host certificate, and curl does not able (curl issue 2793) download intermediate using AIA fields in host certificates
    – filimonic
    Commented Mar 21, 2023 at 20:32
  • But a CA can sign directly the server certificates. The chain MUST be complete (If the CA sign a subordinate CA whose the certificate is absent, this wouldn't work), but this doesn't mean you MUST have 3 certificates (CA, intermediate, server). The fact that there is only an host (self-signed) certificate is important and wasn't clear. This means that either your connection is not secure (you can't trust the server), or the server certificate must be downloaded manually to make curl trusts it. Commented Mar 21, 2023 at 20:41
  • But the chain given by server IS incomplete and contains only host certificate. Virtually, there are 3 certificates in chain - host, intermediate and root. I'm looking for tool that can help curl handling those connections in case when root bundle exists: this tool should get certificate from server, parse AIA records, download full chain and verify this chain against pre-existing root bundle, giving some output for curl (bundle file for this server OR --pinnedpubkey <value>) or null if verifying failed
    – filimonic
    Commented Mar 21, 2023 at 20:46
  • I have tried curl with a self-signed certificate... and it works. All you need is that curl trusts the proposed root. Either throught the /etc/ssl/certs files which are trusted by default, or with a --cacertoption. (Note copying the self-signed certificate in /etc/ssl/certs` doesn't work) Commented Mar 21, 2023 at 21:12

2 Answers 2

5

curl handling of this case depends on the build.

curl can use any of several TLS implementations, determined at compile time, and it should not be much of a surprise that the behavior of the TLS implementation in curl depends on the TLS implementation in curl.

curl with schannel CAN fetch a missing chain cert (not sent by the server, as required by RFCs) using AIA, and validate the result. In addition, curl with schannel will fill-in a missing chain cert if it is in the Windows cert store (just as it will always validate against roots in the Windows cert store), and while I can't easily verify exactly when such chain certs get put in the Windows store, my (Win 10 Home) system, used daily for good bit of web browsing and occasionally for things like curl and powershell iwr, does have about a dozen of them. curl with schannel is only available on Windows, of course. And since you suggest you are using --pinnedpubkey which is not supported with schannel, you may not be using schannel.

OpenSSL definitely does not ever use AIA for this, so curl with OpenSSL, popular on many (but not all) Linux distros, does not either. As Frederic discovered, OpenSSL will fill-in a chain cert (as well as a root) from the file supplied as --cacert, so if you either know or can guess what chain cert(s) you will need, and download them (all) in advance, this will work.

I'm pretty sure NSS itself can do AIA (because Firefox does), but I don't know if and how curl would access or enable it, and the one curl with NSS I can easily test (on Ubuntu 16.04 on WSL, admittedly a bit out of date) does not do it..

I don't know about GnuTLS either way, but curl with GnuTLS on CentOS 7 does not do it.

However, to your real question, I don't of any tool that will do it manually and save the result.

https://www.ssllabs.com/ssltest/ does a very extensive test of SSL/TLS features and compatibility which includes filling-in and validating your cert/chain in a browser-like fashion and displaying the results; if will even display multiple trust paths if it finds them, which browsers don't do. You might be able to scrape those results. https://whatsmychaincert.com/ does only the chain build/correction, and I seem to recall it produces a nice PEM, but I can't verify since the only public test I know of is currently broken (the same one Frederic referenced, incomplete-chain.badssl.com).

Java (more exactly JSSE, the SSL/TLS implementation in Java when using the 'standard' cryptoproviders) can fetch using AIA as part of validation (but not by default, you must enable it) so you could write a simple Java program that connects to a website, validates the cert with AIA.caIssuers enabled, and if successful write out the resulting cert and/or its key for pinning; I'd expect this is only 20-30 lines and if no better solution is provided and I have time later I'll give it a shot. (Or, this being Stack, anyone else can do so.)

1
  • Then a way to make things "automatic", would be to call openssl s_client with the -showcert option, get the certificate, analyse it. If no AIA, use curl normally. If AIA, get its CA Issuers URL. Download this CA, verify it against our trusted root CA, then call curl with this new CA as a root CA... It becomes complex ! Commented Mar 22, 2023 at 20:23
1

curl can handle self-signed certificates.

You just have to type curl --cacert fullchain.pem URL where fullchain.pem is the certificate you have downloaded trought a trusted path.

If you don't feed curl with an appropriate root, curl can't trust the server and you will need the --insecure option to make it happy.

Note that with https://regery.com/en/security/ssl-tools/self-signed-certificate-generator (or other tools), anybody can make a self signed certificate for any server. Then curl have no way to trust a server with a self-signed certificate if you don't give this certificate with an other way.

With other certificates, it is easier, because curl can trust casual root certificates (Verisign, DigiCert, LetsEncrypt... see /etc/ssl/certs/), then curl has a root certificate which is linked to the server certificate. With a server certificate, you have to propose yet an other root.

8
  • I am asking for tool that is able to download fullchain.pem and validate it against manually predownloaded Root CA Bundle automatically to let curl use fullchain.pem in next step and be sure it is already validated.
    – filimonic
    Commented Mar 21, 2023 at 21:11
  • This seems to be the answer you need: serverfault.com/questions/139728/… Afterward, use the certificate with the --cacert option. But It won't validate the certificate... and it can't. Note that in this case, the 'Preloaded Root CA' is also the server certificate. Commented Mar 21, 2023 at 21:21
  • This does not validate downloaded certificate against anything. Here is example I talking about: incomplete-chain.badssl.com - although it has expired, it demonstrates the problem: this certificate is issued by Digicert intermediate, BUT curl will fail validating it because server did not provide digicert intermediate certificate. openssl s_client -connect incomplete-chain.badssl.com:443 -showcerts
    – filimonic
    Commented Mar 21, 2023 at 21:28
  • The problem is not that you miss an 'intermediate' certificate, but that the root is not trusted. (openssl verify prints unable to get local issuer certificate. I have downloaded the CA with a browser and feed it to the command openssl verify -CAfile DigiCertSHA2SecureServerCA.pem your_certificate_chain, and the error is different : the certificates are expired. (This could explain such certificates doesn't match the trusted CA of your client). But the error unable to get local issuer certificate is gone. The CA is here : cacerts.digicert.com/DigiCertSHA2SecureServerCA.crt Commented Mar 21, 2023 at 21:51
  • Not, that with curl --cacert DigiCertSHA2SecureServerCA.pem https://incomplete-chain.badssl.com/, the error change : only the expiration of the certificate (note the difference of format : crt vs pem). I have some DigiCert certificates in my /etc/ssl/certs, but apparently no DigiCertSHA2SecureServerCA. Commented Mar 21, 2023 at 21:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .