alt

NOTE: This vulnerability was not discovered by Shieldmaiden LLC. This post is simply an informative article quoting information already generated by other people. All original data and images can be found in the Resources section of this blog post.

Introduction:

'#cloudbleed' is a newly published vulnerability discovered by Tavis Ormandy (Twitter @taviso). This vulnerability identifies the potential for personal data (PII, passwords, etc) to be parsed and viewed. The depth of this vulnerability is astounding. Anyone third party using cloudflare’s services during this time could potentially have PII+ information from their user base stored in the wild. Cloudflare disclosed the issue, but, the vulnerability appears to be severely downplayed.

Threat Description:

This vulnerability allows for PII and other information to be scraped and compiled into actionable data.

The original post by Mr Ormandy with limited technical details are as follows:


“(It took every ounce of strength not to call this issue "cloudbleed")

Corpus distillation is a procedure we use to optimize the fuzzing we do by analyzing publicly available datasets. We've spoken a bit about this publicly in the past, for example:

https://security.googleblog.com/2011/08/fuzzing-at-scale.html http://taviso.decsystem.org/makingsoftwaredumber.pdf#page=11

On February 17th 2017, I was working on a corpus distillation project, when I encountered some data that didn't match what I had been expecting. It's not unusual to find garbage, corrupt data, mislabeled data or just crazy non-conforming data...but the format of the data this time was confusing enough that I spent some time trying to debug what had gone wrong, wondering if it was a bug in my code. In fact, the data was bizarre enough that some colleagues around the Project Zero office even got intrigued.

It became clear after a while we were looking at chunks of uninitialized memory interspersed with valid data. The program that this uninitialized data was coming from just happened to have the data I wanted in memory at the time. That solved the mystery, but some of the nearby memory had strings and objects that really seemed like they could be from a reverse proxy operated by cloudflare - a major cdn service.

A while later, we figured out how to reproduce the problem. It looked like that if an html page hosted behind cloudflare had a specific combination of unbalanced tags, the proxy would intersperse pages of uninitialized memory into the output (kinda like heartbleed, but cloudflare specific and worse for reasons I'll explain later). My working theory was that this was related to their "ScrapeShield" feature which parses and obfuscates html - but because reverse proxies are shared between customers, it would affect all Cloudflare customers.

We fetched a few live samples, and we observed encryption keys, cookies, passwords, chunks of POST data and even HTTPS requests for other major cloudflare-hosted sites from other users. Once we understood what we were seeing and the implications, we immediately stopped and contacted cloudflare security.

This situation was unusual, PII was actively being downloaded by crawlers and users during normal usage, they just didn't understand what they were seeing. Seconds mattered here, emails to support on a friday evening were not going to cut it. I don't have any cloudflare contacts, so reached out for an urgent contact on twitter, and quickly reached the right people.

https://twitter.com/taviso/status/832744397800214528

After I explained the situation, cloudflare quickly reproduced the problem, told me they had convened an incident and had an initial mitigation in place within an hour.

"You definitely got the right people. We have killed the affected services"

This bug is subject to a 90 day disclosure deadline. After 90 days elapse or a patch has been made broadly available, the bug report will become
visible to the public.”


This vulnerability gave out the following types of information from cloudflare’s customer base (as stated by Mr. Ormandy):


“The examples we're finding are so bad, I cancelled some weekend plans to go into the office on Sunday to help build some tools to cleanup. I've informed cloudflare what I'm working on. I'm finding private messages from major dating sites, full messages from a well-known chat service, online password manager data, frames from adult video sites, hotel bookings. We're talking full https requests, client IP addresses, full responses, cookies, passwords, keys, data, everything.”


Cloudflare used some wordsmithing to play the issue down (Cloudflare’s Blog Disclosure):


“The greatest period of impact was from February 13 and February 18 with around 1 in every 3,300,000 HTTP requests through Cloudflare potentially resulting in memory leakage (that’s about 0.00003% of requests). We are grateful that it was found by one of the world’s top security research teams and reported to us.”


The cloudflare blog post regarding this issue offers a more in depth technical explanation of this incident.

Mitigation Recommendations:

Because this vulnerability is retroactive it is our recommendation that users change their passwords. The new password that you select should be something that you haven’t used before because it may have already been captured. Cloudflare is a major service provider for a huge quantity of websites. Their website states that “Cloudflare makes more than 5,500,000 Internet properties faster and safer. Join today!”

Cloudflare stated that they worked diligently to resolve this issue. The end result is as follows (Cloudflare’s Blog Disclosure):


"The infosec team worked to identify URIs in search engine caches that had leaked memory and get them purged. With the help of Google, Yahoo, Bing and others, we found 770 unique URIs that had been cached and which contained leaked memory. Those 770 unique URIs covered 161 unique domains. The leaked memory has been purged with the help of the search engines. We also undertook other search expeditions looking for potentially leaked information on sites like Pastebin and did not find anything.”


This means that anyone that pulled down that information can retroactively compile PII form, passwords, etc. from it.

Conclusion:

This vulnerability is insane. It is described as being worse than SSL heartbleed because of its retroactive nature. Anyone that pulled that data down and stored it will still have access to all of that private data. Which means that any information that customers sent over https to sites using cloudflare should consider their data public and work towards securing their accounts.

Resources: