Cloudflare has become the latest web infrastructure giant to collapse in the span of a month, replacing entire sites, including X, ChatGPT, Spotify, Canva, and even the outage-tracking DownDetector, with an error message for hours this morning. It’s the latest in a string of outages that Mehdi Daoudi, CEO and co-founder of the internet performance monitoring platform Catchpoint, says should be a “wake-up call” for companies.
“Everybody’s putting all their eggs in one basket, and then they’re surprised when there is a problem,” Daoudi says. “It’s on the company’s side to make sure that they have redundancy and resiliency.”
The outage comes after issues affecting Microsoft Azure and Amazon Web Services occurred within just one week of each other, bringing down large chunks of the internet that rely on major providers to keep their websites running. Cloudflare similarly powers a sizable part of the internet. It keeps websites online with its content delivery network, while offering several other services, including DDoS attack protection and DNS. Last year, the company said around 20 percent of the web runs through Cloudflare’s network. It also serves 35 percent of companies on the Fortune 500 list, in addition to “millions” of other customers.
Cloudflare’s speedy performance and security record make it a popular choice for websites across the globe, but this latest outage draws attention to just how concentrated the web infrastructure industry has become. After the AWS outage took down the secure messaging app Signal, the service’s president, Meredith Whittaker, said the company didn’t have any other choice but to use a major cloud service provider to run on. “The entire stack, practically speaking, is owned by 3-4 players,” she wrote.
“Even small deviations can have outsized consequences.”
But even with companies relying on just a few web infrastructure providers, the last chain of outages makes it clear that they need a backup plan. “Outages will be here, and they’re just going to keep happening more frequently. The blast radius will keep growing,” Daoudi tells The Verge. “The question is, what are you doing about it?”
Though Microsoft and AWS linked their outages to issues related to DNS — a system that translates website domain names into IP addresses — Cloudflare traced its outage to a single file. “The root cause of the outage was a configuration file that is automatically generated to manage threat traffic,” Cloudflare spokesperson Jackie Dutton said. “The file grew beyond an expected size of entries and triggered a crash in the software system that handles traffic for a number of Cloudflare’s services.”
It may seem absurd that a file issue like this could bring down swaths of the internet, but for companies as large as Cloudflare, it can happen. “When you operate infrastructure at Cloudflare’s scale, even small deviations can have outsized consequences,” Rob Lee, the chief of AI and research at the SANS Institute, tells The Verge. “These platforms are built for speed, so anything that delays or halts decision making can cascade quickly. In high performance environments, a millisecond delay can become a complete traffic stoppage.”
According to Lee, a configuration file like the one Cloudflare describes “drives routing security policies, load balancing decisions, and how traffic is distributed globally.” If the file suddenly increases in size, “it can trigger slower parsing, memory issues, CPU contention, or logic failures inside the systems that rely on it,” Lee adds.
AWS similarly blamed “faulty automation” for setting off a chain of issues that led to its most recent widespread outage — the kind of error that’s bound to happen again. “Are you going to complain about it every time Cloudflare sneezes?” Daoudi says. “Or are you going to build around it?”