Scaling

What does backpressure tell you?

Backpressure is the system stating its true capacity. The design question is who receives the message: a queue, a caller, or a customer.

By TIS Partners · · 2 min

Backpressure has a bad reputation it does not deserve. When a system pushes back (slows accepts, sheds load, returns "try later"), it is telling the truth about its capacity. The pathologies come from systems designed to never say it, because the truth then arrives anyway, unscheduled, as a collapse rather than a signal.

Duotone illustration of flow narrowing against a gate with pressure marks upstream

What is the signal, exactly?

Arrival rate exceeding service rate, somewhere. Little's law makes the bookkeeping exact: items in the system equal arrival rate times time inside. When work arrives faster than it leaves, something must grow: a queue's depth, a latency, a memory footprint, an error rate. Backpressure is choosing which, on purpose, and making the choice visible upstream. No backpressure means the choice happens anyway, invisibly: the unbounded queue grows until memory ends, the reservoir with no policy, the retry storm doubles arrivals precisely at saturation.

The classic public incident in this family remains Slack's January 4th, 2021 outage: the year's first Monday, load arriving faster than scaling could, and failure amplifying through the layers that had no way to say "slower, please." Every organization owns a smaller version of that Monday.

Who should receive the message?

This is the actual design decision, and there are only three addresses. The infrastructure can absorb it (buffers, autoscaling) which works until the burst outlives the buffer; absorption is a loan, not forgiveness. The caller can receive it (429s, bounded retries with jitter, the builders-library discipline), which keeps the truth moving toward whoever can act on it. Or the customer can receive it, shaped: the queue-position page, the degraded mode, the "reports are delayed" banner. Product people hate designing for this address until the day it saves the quarter; a graceful "later" retains customers that a spinning timeout does not.

The worst address is the default one: nowhere. Systems that accept everything and fail collectively deliver the message to all three addresses at once, at maximum amplitude, with an incident number attached.

What would we have you instrument?

Queue depths with alerts on trend, not threshold; depth rising at constant arrivals means service rate fell, which is an early warning nothing else provides. Age of oldest item, because depth alone hides a stuck partition. And the shed counter: how often you said no. A shed counter at zero forever means either overprovisioning or, more commonly, that the system has never been asked its limit and the first exam is scheduled by whatever the capacity model missed.

Backpressure is capacity speaking. Design the sentence it gets to say, the audience that hears it, and the dashboard that records it, and load stops being a thing that happens to you and becomes a thing you meter. Systems, like practices, are healthiest when they can decline work out loud.