I just renewed a public X.509 certificate issued by Multicert in a site/vhost in a RedHat 6.2 VM running Apache 2.2. Let's call it https://www.multicert.com . My client machine visiting the site with Chrome is running Debian 9.

What surprised me is that the certificate is giving an approved/green in Firefox Quantum 60.2.0esr (64 bit) and in Safari too, however Chrome 69.0.3497.92 is now complaining the site is not secure (while before with the old certificate it was ok).

I checked the Apache configuration, and all seems fine. I also tripled checked the X.509 certificate chain and the root, and all seems Ok.

We also have another public certificate issued at the same time for a similarly configured site, however, it is issued Comodo and not Multicert, and in this site Chrome works fine with the certificate, let's call it https://www.digicert.com

If I revert to the old certificate, Chrome works again, however I cannot leave it like that as it will be probably revoked tomorrow, and expire in a couple of days nonetheless.

The only change we noticed in the site with the Comodo certificate, is in Chrome, when clicking on the certificate lock->Certificate-details, we have new field under Extensions with the identifier OID.


What is happening here?


Given the OID ., I found out a relevant Let's Encrypt article Engineering deep dive: Encoding of SCTs in certificates

Let’s Encrypt recently launched SCT embedding in certificates. This feature allows browsers to check that a certificate was submitted to a Certificate Transparency log.

I started making some investigations in the subject and ended up finding out Google made mandatory Certificate Transparency in Chrome for all kind of X.509 certificates from the 1st of May of 2018.

From Certificate Transparency Enforcement in Google Chrome

This notice is being sent to all CA Operators known to the Common CA Database [1] and applies to CAs that are currently, or may in the future be, trusted on platforms on which Chrome operates (Mozilla NSS, Microsoft Windows, Apple iOS/macOS, Google ChromeOS, and Android).

We’re writing to reiterate the upcoming April 2018 Certificate Transparency (CT) enforcement in Chrome. As was first announced [2] at the CA/B Forum, followed by announcements [3] on the mozilla.dev.security.policy forum, and later updated [4] in the referenced ct-policy forum, Chrome will start enforcing that all TLS certificates issued after April 2018 comply with the Chromium CT Policy [5] in order to be trusted.

Since January 2015, Chrome has required that Extended Validation (EV) certificates be CT-compliant in order to receive EV status. In April 2018, this requirement will be extended to all newly-issued publicly-trusted certificates - DV, OV, and EV - and certificates failing to comply with this policy will not be recognized as trusted when evaluated by Chrome. Certificates issued from locally-trusted or enterprise CAs that are added by users or administrators are not subject to this requirement.

What is happening and when?

Chrome will require that all TLS server certificates issued after 30 April, 2018 be compliant with the Chromium CT Policy. After this date, when Chrome connects to a site serving a publicly-trusted certificate that is not compliant with the Chromium CT Policy, users will begin seeing a full page interstitial indicating their connection is not CT-compliant. Sub-resources served over https connections that are not CT-compliant will fail to load and will show an error in Chrome DevTools. CAs are strongly encouraged to work with their customers to ensure their TLS certificates are ready to comply with the Chromium CT Policy via any of the three means specified in RFC 6962 Section 3.3 [6] before the end of March to ensure that any issues with deploying CT support are resolved in advance of the enforcement deadline. These changes will be rolled out to Desktop platforms first, which include macOS, Windows, Linux, and ChromeOS.

In order to accommodate the unique needs of certain enterprises, there will be Chrome policies to disable CT enforcement on managed devices and for managed users that have signed-in to Chrome on their personal devices. In addition to the existing ability to disable CT enforcement by URL [7], Chrome will add a policy that allows organizations to disable CT enforcement for CAs that only issue certificates to that organization.

From Chrome Requires CT after April 2018

Chrome will require that all TLS server certificates issued after 30 April, 2018 be compliant with the Chromium CT Policy.” This means SSL/TLS certificates must be CT qualified by meeting one of the following criteria:

  • A signed certificate timestamp (SCT) from a log qualified at the time of check is presented via the TLS extension OR is embedded within a stapled OCSP response, where there is at least one SCT from a Google Log, qualified at the time of check and at least one SCT from a non-Google Log, qualified at time of check.

  • An embedded SCT from a log qualified at the time of check is presented, where there is at least one embedded SCT from a Google Log, at least one Embedded SCT from a non-Google Log and there are the minimum number of embedded SCTs.

So from Certificate Transparency, an introduction

Signed Certificate Timestamp

Certificates will typically be submitted to CT logs by the CA that issued them but it is also possible for a site owner to submit their own certificates to a log too. An SCT is the response from a Certificate Log when you submit a certificate which is essentially a promise that the certificate will be added to the log in a given amount of time. It's this SCT that we have to deliver to clients when they make a TLS connection to our site and there are 3 ways that we can do this.

x.509v3 Extension

The preferred method to deliver an SCT for a site operator is via an X.509v3 extension. This means that your CA will embed the SCT in your certificate before they sign it and issue it to you so absolutely no work or setup is required on your part. The CA will construct your certificate and prior to the last step of signing it they will submit it to the CT log as a precertificate. The log will respond with the SCT for the precertificate, the CA will embed that in the certificate and then sign it, ready for issuing to you.

TLS Extension

The SCT can also be delivered via a TLS extension from the host to the connecting client. This takes place during the TLS handshake using an extension called signed_certificate_timestamp but does require the host to update their server and configure it to deliver the SCT. The benefit here is for the CA, who doesn't have to change how they issue certificates , but waiting for reliable server implementations of the SCT TLS extension might not be great for site operators.

OCSP Stapling

I'm a fan of OCSP Stapling and it's usually used to deliver revocation information about our certificate but it can also be used to deliver the SCT from the CA to the site operator. There's a benefit for the CA again as they don't have to change their issuance process, they simply sign and issue the certificate as normal and make the SCT available via OCSP. Again though, this is a problem for us as our server needs to be able to reliably OCSP Staple and include the SCT to the client during the handshake.

So it turns out about my two sites, one (example: www.digicert.com) is having the SCT X.509 extensions directly in the certificate, and needs no setup modifications on the server side.

On the other site (example: multicert.com), the CA operator choose to use X.509 stapling, and as such, the Apache web server needs configuration changes.

I also located an article at Digicert about OCSP stapling for Apache

So to get the site with OSCP stapling work, I need to:

  • have an Apache version greater than 2.3.3
  • the VM having communication to the CA OCSP server
  • adding to the vhost:

    Outside the < virtualhost...> directive

    SSLStaplingCache shmcb:/tmp/stapling_cache(128000)

    Inside the < virtualhost...> directive

    SSLUseStapling on

    And restart Apache.

After those changes in the site with the Multicert issued X.509 certificate, Chrome is saying the certificate(s) are valid in both sites.

See also Chrome Linux is not showing an X.509 Extension, SCT

More technical details

I was also asked how the the time threshold of 1 May of 2018 was imposed, and what would be happening to the old certificates. Lacking more palpable details online, I downloaded the Chromium source from https://chromium.googlesource.com/chromium/src/ with the command:

git clone https://chromium.googlesource.com/chromium/src 

For those interesting in the Certificate Transparency functionality, the more interesting directory seems to be components/certificate_transparency/ and the most interesting doc file net/docs/certificate-transparency.md

Interesting excerpts From net/docs/certificate-transparency.md


Certificate Transparency (CT) is a protocol designed to fix several structural flaws in the SSL/TLS certificate ecosystem. Described in RFC 6962, it provides a public, append-only data structure that can log certificates that are issued by certificate authorities (CAs). By logging certificates, it becomes possible for the public to see what certificates have been issued by a given CA. This allows site operators to detect when a certificate has been issued for their domains, allowing them to check for unauthorized issuance. It also allows browsers and root stores, and the broader community, to examine the certificates a CA has issued and ensure that the CA is complying with their expected or disclosed practices. >


We say that a certificate supports Certificate Transparency if it comes with CT information that demonstrates it has been logged in several CT logs. This CT information must comply with the Certificate Transparency in Chrome policy. We sometimes refer to a site that "supports" CT as using a certificate that is "CT qualified" or "disclosed via CT."

Chrome Policies

Chrome has gradually required Certificate Transparency for more and more publicly-trusted certificates over the past few years.

  • Since 1 January 2015, Chrome has required that all Extended Validation certificates be disclosed via Certificate Transparency. Certificates that were not properly disclosed would be stripped of their EV status, but no warnings would be shown to visitors to sites that did not comply.

  • Since 1 June 2016, Chrome has required that all new certificates issued by the set of root certificates owned by Symantec Corporation are disclosed via Certificate Transparency. Certificates that were not disclosed, or which were not disclosed in a way consistent with RFC 6962, would be rejected as untrusted.

  • For all new certificates issued after 30 April 2018, Chrome will require that the certificate be disclosed via Certificate Transparency. If a certificate is issued after this date and neither the certificate nor the site supports CT, then these certificates will be rejected as untrusted, and the connection will be blocked. In the case of a main page load, the user will see a full page certificate warning page, with the error code net::ERR_CERTIFICATE_TRANSPARENCY_REQUIRED. If you receive this error, this indicates that your CA has not taken steps to make sure your certificate supports CT, and you should contact your CA's sales or support team to ensure you can get a replacement certificate that works.

Domain Privacy

Supporting CT by disclosing the certificate to a CT Log means that the full contents of the certificate will be publicly accessible and viewable. In particular, this means that the domains a certificate are for will be included in the Certificate Transparency log, as well as the organization they are affiliated with, if they are validated to a level higher than Domain Validation or issued from an organization-specific CA.

Note: Interestingly enough, RFC 6292 is defined as experimental

As for the time starting date of 2018-May-1, there are specific hard coded instances in the Chromium code (that is common to Chrome), defining the cut-off date that will be present on the Chrome/Chromium code in the transitioning years. So that explains the different behaviour in certificates issue before 1-May-2018.

From services/network/network_context.cc:

1525         // For old certificates (issued before 2018-05-01),
1526         // CheckCTRequirements() may return CT_NOT_REQUIRED, so we check the
1527         // compliance status here.
1528         // TODO(https://crbug.com/851778): Remove this condition once we require
1529         // signing certificates to have CanSignHttpExchanges extension, because
1530         // such certificates should be naturally after 2018-05-01.
1531         if (ct_verify_result.policy_compliance ==
1532                 net::ct::CTPolicyCompliance::CT_POLICY_COMPLIES_VIA_SCTS ||
1533             ct_verify_result.policy_compliance ==
1534                 net::ct::CTPolicyCompliance::CT_POLICY_BUILD_NOT_TIMELY) {
1535           ct_verify_result.policy_compliance_required = true;
1536           break;
1537         }

From components/certificate_transparency/chrome_ct_policy_enforcer.cc:

217   // ... AND there is at least one embedded SCT from a Google Log once or
218   //   currently qualified;
219   // AND there is at least one embedded SCT from a non-Google Log once or
220   //   currently qualified;
221   // ...
222   //
223   // Note: This policy language is only enforced after the below issuance
224   // date, as that's when the diversity policy first came into effect for
225   // SCTs embedded in certificates.
226   // The date when diverse SCTs requirement is effective from.
227   // 2015-07-01 00:00:00 UTC.
228   const base::Time kDiverseSCTRequirementStartDate =
229       base::Time::UnixEpoch() + base::TimeDelta::FromSeconds(1435708800);
230   if (issuance_date >= kDiverseSCTRequirementStartDate &&
231       !(has_embedded_google_sct && has_embedded_nongoogle_sct)) {
232     // Note: This also covers the case for non-embedded SCTs, as it's only
233     // possible to reach here if both sets are not diverse enough.
234     return CTPolicyCompliance::CT_POLICY_NOT_DIVERSE_SCTS;
235   }

Adenda: based on something I discovered after doing this investigation, and that is detailed on: Chrome Linux is not showing an X.509 Extension, SCT

SCT extension for https://www.digicert.com


OCSP definition for https://www.multicert.com


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.