What Can We Learn from the Cloudflare Leak?

Cloudflare calls itself the “web performance and security company,” so it was a serious blow to its reputation when researchers discovered that it had a security bug that made sites’ data visible on other sites. What was really disturbing was that supposedly secure data from HTTPS requests leaked out this way. Passwords, session cookies, credit card information, and other sensitive data simply showed up in random places.

Google researcher Tavis Ormandy discovered this problem on February 17, and tech media have attached the name “Cloudbleed” to it. Cloudflare provides services to millions of websites, and any of them could have suffered a loss of confidential data. Many of them have urged users to change their passwords. The risk to any individual is low, but the effect was so widespread that personal data could have been stolen from a significant number of people.

Cloudflare has fixed the bug, but the leaked data could still be lurking in the caches of search engines and edge servers, and data thieves now know to look for it.

Cloudflare’s incident report explains that the problem stemmed from a buffer overrun bug. For efficiency reasons, low-level system software is often written in programming languages, such as C, which don’t automatically guard against accessing memory structures outside their limits. An HTML parser had a bug of this type, resulting in its picking up data from whatever was past the end of a memory buffer. It could be anything, and sometimes it was private data from another website.

The risk in third-party services

Any website can have bugs in its software that open security holes. That’s one reason HTTPS connections aren’t 100% secure. Old versions of SSL (TLS) have problems. The “Heartbleed” bug in older versions of the widely used OpenSSL software showed it was possible to exploit the weaknesses. The latest version fixes the problem, but there’s no guarantee that it’s completely bug-free. Many websites still use old versions of OpenSSL, with known weaknesses.

When a site uses a third-party service such as a caching proxy or a content delivery network, it can gain or lose security. A top-quality CDN has better security measures than most do-it-yourself sites, and it filters requests to the sites’ servers. It can absorb DDoS attacks that would kill a one-machine server. Cloudflare features a web application firewall (WAF) that protects sites at the application level from many kinds of attacks.

This comes at a price, though.

To get the full range of services from Cloudflare, a website has to hand over its most precious secret: its private SSL key. Without that datum, Cloudflare couldn’t do anything with HTTPS requests and responses but pass them through. It wouldn’t be able to see anything except what server and port number they were going to.

The fact that the breach included HTTPS data underscores this issue. If Cloudflare didn’t have sites’ private keys, it could never have leaked passwords that were properly sent through HTTPS. By the same token, it couldn’t have provided a useful WAF to protect servers that use secure communication. Sharing a private key with a CDN creates a potential risk, even if there’s an overall gain in security.

Vulnerability to governments

However, giving a CDN a site’s private key opens up one serious hole, which no software can guard against. A government can demand it, compel the CDN to stay silent, and have access to all of the site’s SSL transactions. Government agents can spy on it indefinitely, and the site’s owners won’t have a clue that it’s happening.

In the United States, a National Security Letter can accomplish this. Anyone who receives one isn’t allowed to say anything about it or challenge it in an open court hearing. The Electronic Frontier Foundation has called the power to issue them “one of the most frightening and invasive” surveillance power created by the PATRIOT Act.

Cloudflare has received at least two NSLs and possibly more. The FBI could have compelled it to turn over customers’ private keys and not tell them. In a similar case, the FBI tried to compel Lavabit, a confidential email service, to turn over keys that would give it access to every user’s private mail, even though it was just after Edward Snowden. Founder Ladar Levison was under a gag order not to disclose this until recently.

Other countries have similar or worse issues. The UK’s Investigatory Powers Act gives law enforcement the authority to make telecommunication companies break their encryption. They would be under a compulsion of secrecy comparable to a National Security Letter. In the truly authoritarian states, the situation is even worse, with privacy being virtually non-existent.

How many websites do governments have access to, without their knowledge, because CDNs had to give up their private keys? There’s no way to know.

The OrangeWebsite Difference

At OrangeWebsite we take your privacy seriously. We don’t share our private keys, or yours, with third-party services. Government agencies in North America or Europe can’t demand anything from us. We maintain state-of-the-art server security, performing regular security audits and keeping system software up to date. Optional two-factor authentication is available.

Nomad Capitalist has called Iceland the best host country for data privacy. The Icelandic Modern Media Initiative, passed by our Parliament in 2010, commits the country to freedom of information and expression. We allow anonymous registration, so that even torture or telepathy wouldn’t get us to disclose your identity. Contact us to learn how to set up a secure, censorship-free website.