SANS ISC: DigiNotar breach - the story so far - SANS Internet Storm Center

• SANS Site Network
  • Current Site
  • SANS Internet Storm Center
  • Other SANS Sites Help
  • Graduate Degree Programs
  • Security Training
  • Security Certification
  • Security Awareness Training
  • Penetration Testing
  • Industrial Control Systems
  • Cyber Defense Foundations
  • DFIR
  • Software Security
  • Government OnSite Training

SANS ISC InfoSec Forums

I've been following the DigiNotar story as it evolved for a few days now with growing concern and increasing alarm.

I'm by far not privy to the inside information to be able to really assess and audit the situation, so this is purely based on what is publicly known. Being a Dutch native speaker I have access to what the press in the Netherlands writes about it with the subtle nuances that an automated translation will not capture. I do lack the resources to independently double verify everything and as such some errors might still be in it, consider this a best effort at creating some overview and leading up to conclusions with the limited information that is available.

If we do attract the attention of DigiNotar and/or Vasco: please do contact us, we'd love to talk to you and get more information!

So who is DigiNotar and what do they do when all is normal?

DigiNotar is a CA. They sell SSL certificates, also the EV kind.

But there is more that's mostly of interest to those in the EU or the Netherlands only:

They are also (I'm simplifying a bit, I know) an accredited provider in the EU and provide qualified certificates and approved SSCDs to customers to create digital signatures that -by law- in the EU are automatically considered to be qualified digital signatures and as such they are automatically equivalent to manual signatures. This status forces regular 3rd party audits against the relevant Dutch law and standards such as ETSI TS 101 456.

They also provide certificates services under the PKIOverheid umbrella in the Netherlands. This has even more and stricter rules. e.g. Things that are suggested in the ETSI standards, but not mandatory, can become mandatory for PKIOverheid.

DigiNotar is a 100% daughter company of Vasco (since Jan 2011), so if you see Vasco sometimes doing things like press releases regarding the incident, that's why. 

So what do we know in a chronological order ?

• Dating back as far as May 2009, the portal of DigiNotar has been defaced, these hacks remained in place till this week after f-secure exposed them in their blog.
  Source: f-secure blog
• On July 10th 2011 a certificate was issued with a CN of *.google.com by DigiNotar
  Source: pasted certificate
• In July 2011 "dozens" of fake certificates were issued by intruders -most likely Iranian, but that remains to be proven-.
  Source: Jan Valcke, Operational director at Vasco in an interview with "webwereld" [in Dutch]
  The list of fake certificates appears to include certificates for mozilla, tor, yahoo, wordpress and baladin.com, but does not include any financial institutes.
  Source: nu.nl article [in Dutch] would love a second or more authoritative source for this.
• On July 19th 2011, DigiNotar detected the incident and supposedly the majority of these certificates were revoked. At least one, possibly more certificates were missed in this process.
  Source: Jan Valcke, Operational director at Vasco in an interview with "webwereld" [in Dutch]
  There's a bit of a bad feeling with this claim, see further.
• On an unknown date, an unknown external auditor did not catch the fraudulent certificate for *.google.com. as well as any others that might be missed as well.
• On Aug 28th 2011, (some sources claim 27th) a user from Iran posted on a forum using Chrome was warned by his browser the certificate was not to be trusted.
  Source: Forum post
  Chrome does additional protections for gmail since chromium 13.
• On Aug 29th 2011, the *.google.com certificate was revoked by DigiNotar
  This can be seen in the n at http://service.diginotar.nl/crl/public2025/latestCRL.crl [do not click on this URL, most browsers "understand" CRLs], see further.
• On Aug 29th 2011, the response from Google and the other browser makers came:
  • Google
  • Microsoft blog and advisory
  • Firefox
  Basically the "sh*t hit the fan" as the browser vendors are pulling the plug on DigiNotar and not trusting their processes anymore.
• On Aug 30th 2011, Vasco issued a press release reporting the incident.
• On Aug 30th 2011, various claims of both Vasco, and the Dutch government try to stress that the activities of DigiNotar under the PKIOverheid root were not affected. Some arguments used in the press such as that the root certificate of PKIOverheid is not at DigiNotar (they have an intermediate) are obvious and irrelevant.
• On Aug 31th 2011, it is confirmed security company Fox-IT is performing a forensic audit of the systems of DigiNotar. Results are expected next week at the earliest.
  Source: webwereld article [in Dutch]

Analysis of the CRLs

DigiNotar claims all breaches were under the "Public 2025 Root" ref [in Dutch]. What "root" does in there is somewhat unclear to the technical inclined mind, and the "public 2025" just seems to be some sort of internal name. Let's assume they meant the fraudulent certificates all were signed by the same intermediate.

The CRL indicted in the fraudulent *.google.com certificate does indeed point in the same "public 2025" direction, so let's get that CRL:

$ wget http://service.diginotar.nl/crl/public2025/latestCRL.crl

Let's make this file human readable:

$ openssl crl -text -inform DER -in latestCRL.crl >/tmp/t

And let's verify there is indeed the Serial Number in there of the *.google.com fake certificate we found on pastebin:

$ grep -i "05e2e6a4cd09ea54d665b075fe22a256" /tmp/t
    Serial Number: 05E2E6A4CD09EA54D665B075FE22A256

So yes, it's revoked. Getting the other relevant lines (it means first figuring out how many, but I skip the boring part).

$ grep -i -A4 "05e2e6a4cd09ea54d665b075fe22a256" /tmp/t
    Serial Number: 05E2E6A4CD09EA54D665B075FE22A256
        Revocation Date: Aug 29 16:59:03 2011 GMT
        CRL entry extensions:
            Invalidity Date:
                Aug 29 16:58:47 2011 GMT

So that checks out nicely. [One should of course check that all signatures are valid everywhere]

Unfortunately one can only see the Serial Number of the certificates revoked, not the more juicy fields like the CN or so that would allow to see what and when other (fake) certificates were revoked.

But since we have the revocation date, maybe we can see the peak where they revoked the fraudulent certificates. I know the nature of revocation and any other work in a CA/RA can be highly cyclic with huge peaks in it, and I know not to worry about any revocation as such, users loosing control over a certificate happens all the time.

So let's see revocation activity in July 2011 split out per day:

$ grep "Revocation Date:" /tmp/t | sed 's/^.*Date: //' | sed 's/..:..:.. //' 
|sed 's/GMT//' | sort -n | uniq -c  | grep 'Jul .* 2011'
   1 Jul  1 2011
   3 Jul  4 2011
   3 Jul  5 2011
   6 Jul  6 2011
   6 Jul  7 2011
   1 Jul  8 2011
   2 Jul 11 2011
   6 Jul 14 2011
   1 Jul 15 2011
   1 Jul 18 2011
   2 Jul 19 2011
   1 Jul 20 2011
   1 Jul 21 2011
   3 Jul 22 2011
   3 Jul 26 2011
   7 Jul 28 2011
   5 Jul 29 2011

Uhmm, where is the "dozens" on July 19th ?

Since the *.google.com one was made on Jul 10th, there is no dozens neither before nor shortly after the 19th.

They might have been added to another CRL, hard to say as DigiNotar does not allow directory listing and doesn't have an easy to find list of CRLs they publish either.

Still, even if we look at the "normal" workload in 2011:

$ grep "Revocation Date:" /tmp/t | sed 's/^.*Date: //' | sed 's/..:..:.. //' 
|sed 's/GMT//' |grep 2011| sed 's/ .. 2011//'| sort -n | uniq -c
  93 Apr
  34 Aug
 112 Feb
 144 Jan
  52 Jul
  18 Jun
 118 Mar
 118 May

We see that the Jun/Jul and Aug months are very light on revocations. [Note that August was not yet complete in GMT time when I downloaded the CRL file].

I know my sed, grep commands could be optimized to save a few CPU cycles, but this isn't a unix lesson.

I'd love to see the "dozens" of revocations around July 19th in a DigiNotar CRL, but I simply cannot find them.

So what's the known impact right now:

• If you're a general Internet user: you're unlikely to see much impact, maybe you'll run into a website with a DigiNotar certificate somewhere that will now warn the certificate is not trusted anymore.
  The longer term impact will still have to manifest itself, and for sure breaches such as these will prompt thinking of other solutions.
  If the add-ons of Mozilla were indeed attacked using a MitM approach, impact might be widespread, but that becomes somewhat less likely.
• If you're a user in Iran, and had something to hide from your government, odds are you're in trouble with your government.
• Tor users: I don't know enough about how tor works, to be determined, but it sounds bad.
• If you're a stakeholder in the Dutch PKIOverheid, well then I'd be careful with the preliminary "all is well" messages, I know PKIOverheid a little bit and I know it's one of the strictest things to get a certificate from, but never say never till it's proven. I do understand the need to keep confidence in the system, but that is also achieved by first investigating before saying there is no problem based on false logic and/or irrelevant data.
• You're a customer of DigiNotar: DigiNotar lost the trust from the browser makers, how permanent that is is too soon to say, but it's a big unprecedented dent.
  If you're a PKIOverheid customer that leaves you a bit more breathing room, and there are 6 more providers to migrate to, and apparently no urgent need to do so.
  Other customers seem to have been offered to migrate to PKIOverheid, but the stringent requirements there might be too much for some, so your best option might be to seek another provider, if you have not done so already.
• If you're a CA or RA, this is yet another big wake-up call. If you're a 3rd party auditor of said, it's the exact same thing. CAs are now a target.

What is the biggest thing we all lack to better see what impact there is/was ?

• Full list of fraudulent certificates (CN, SN fields at the very least)
• Clarity on when each certificate was created and when it was revoked
• I for one would love to know who that external auditor was that missed defaced pages on a CA's portal, that missed at least one issued fraudulent certificate to an entity that's not a customer, and what other CAs and/or RAs they audit as those would all loose my trust to some varying degree. This is not intended to publicly humiliate the auditor, but much more a matter of getting confidence back into the system. So a compromise that an unnamed auditor working for well known audit company X is now not an auditor anymore due to this incident is maybe a good start.
• Clarity over what was affected by the hackers, a full report would be really nice to read. Special attention should be given to explain how it is sure PKIOverheid, the qualified certificates etc. are for sure not affected and how privacy of other customers e.g. was affected. Similarly the defacements should be covered in detail as well as how they could be missed for so long.

Obviously it's unlikely we'll get all those details publicly, but the more we get the easier it will be to keep the trust in the SSL "system" in general and more specifically in DigiNotar.

Glossary

• CA: Certificate Authority
• CN: Common Name, in case of a SSL certificate for a web server this contains the name of the website, can be a wildcard as well in that case.
• CRL: Certificate Revocation List a machine readable list of revoked certificates, typically published over http. Contains the Serial Numbers (SN) of the revoked certificates along with some minimal supporting data.
• "dozens" used in my text above is a somewhat freely translation of the Dutch "tientallen", literally, "multiple tens"
• ETSI TS 101 456: A technical specification on "policy requirements for certification authorities issuing qualified certificates"; used as a norm in audits of said providers.
  Can be freely downloaded from ETSI: version 1.4.3.
• EV: extended validation: essentially the same SSL certificate, but with a slightly stricter set of rules on issuing. Most browsers render something like the URL in the address bar in a green color when they see such a certificate
• PKIOverheid: a PKI system run under very strict requirements by and for the Dutch Government. There are 7 providers recognized to deliver certificates under a root certificate held by the Dutch government. This PKI not only issues certificates to (web) servers, but also to companies and individuals to do client authentication against government websites as well as provide means to create qualified digital signatures.
• RA: Registration Authority
• SN: Serial Number
• SSCD: Secure Signature Creation Device. Mostly a smartcard or smart USB token that holds key pairs used for signing and protects the secret keys

--
Swa Frantzen -- Section 66