Architecture
Can you operate the system while the dashboard is down?
Cloudflare logged three incidents in one day and kept serving traffic through all of them. A briefing on control planes, data planes, and which one your runbook secretly needs.
By TIS Partners · · 2 min
Last week Cloudflare's status history logged three incidents in roughly twenty-four hours: a network problem in Ashburn, elevated errors for Durable Objects and its downstream products in APAC, and a Workers KV problem that took out the company's own dashboard for an hour and a half. Through the third one, the edge kept serving traffic. Customers lost the ability to look at the system and to change it, not the system itself.

That distinction has a name, and we look for it in every review: the data plane is the part that does the work, the control plane is the part that configures, observes and commands the work. The week's useful lesson is that they failed separately. That is not luck; it is a property somebody designed, tested, and paid for.
What does the data plane do when the control plane dies?
The property to ask for is static stability: the serving path keeps running on its last known configuration when the configuration source goes away. A CDN edge that cannot reach its config store should serve yesterday's rules, not refuse traffic. The failure we find more often is the opposite: a proxy that re-resolves its routing from a live registry on every deploy, a cache that treats "cannot refresh" as "cannot serve," a feature flag client that fails closed on the checkout path. Each of these promotes the control plane onto the request path, which means its availability now multiplies into yours.
Can you change the system without the console?
An hour without a dashboard is survivable if nothing needs changing. Incidents are precisely when something needs changing, and the console outage and the incident will not schedule themselves apart. So the review question is concrete: name the last time anyone exercised the second path. An API token with the right scopes, in a place reachable during the outage, used recently enough that someone knows its syntax, is a break-glass procedure. The same token, unexercised for a year, is a hope.
Does your tooling share the fate of what it manages?
The sharpest version of the pattern: Cloudflare's dashboard was impaired by a Workers KV incident, which is to say the vendor's control plane leaned on one of the vendor's own products. Most estates we review contain the same loop at smaller scale. The deploy system runs on the cluster it deploys to. The status page sits behind the load balancer whose status it reports. The runbook lives in the wiki that shares the failing single sign-on. None of these are absurd choices; each was the cheap and consistent option at the time. The review's job is to find which loops exist and price which ones are worth cutting, because cutting one costs a second system, and a second system is a real cost that should be spent on the loops that matter.
The week's score, for the record: three incidents, one vendor, zero sustained damage to the serving path. Read your own estate against that line. If the dashboard going dark for ninety minutes would cost you the ability to serve, observe, or intervene, the diagram has one plane where there should be two.