Measuring What Matters: The DORA Metrics

Module 3 introduced Measurement as one of the five CALMS letters, with a warning: measuring the wrong things is worse than measuring nothing, because it optimizes behavior toward a number instead of toward the outcome the number was supposed to represent. So what should a team actually measure? This isn't a matter of opinion — it's one of the more rigorously researched questions in software engineering, thanks to the DevOps Research and Assessment (DORA) team, originally an independent research group (later acquired by Google), which has surveyed tens of thousands of engineering professionals over several years to find out which practices and metrics actually correlate with organizational performance.

Their research converged on four metrics — now universally referred to as "the DORA metrics" — that, taken together, capture both how fast a team delivers software and how safely it does so. The pairing matters: any one of these numbers alone can be gamed or misleading (Module 3's postmortem-style anti-pattern of splitting one change into many fake "deploys" to inflate a count), but together, they resist gaming, because speed metrics and stability metrics pull in opposite directions if a team tries to cheat one at the expense of the other.

The four metrics

1. Deployment Frequency — how often an organization successfully releases to production. Elite performers deploy on demand, often multiple times a day.
2. Lead Time for Changes — how long it takes from a code commit to that code running successfully in production. For top performers, this is measured in less than a day, not weeks.
3. Change Failure Rate — what percentage of deployments to production result in a degraded service requiring remediation (a rollback, a hotfix, an incident). Elite performers keep this under roughly 4%.
4. Time to Restore Service (MTTR) — when a failure does happen, how long it takes to restore service. Elite performers typically recover in under an hour.

Notice the shape of this list: two speed metrics (Deployment Frequency, Lead Time) and two stability metrics (Change Failure Rate, Time to Restore). That's deliberate. A team that only tracked deployment frequency could hit an impressive number by deploying carelessly and breaking things constantly — the change failure rate would immediately expose that trade-off. A team that only tracked change failure rate could hit zero by never deploying anything — deployment frequency would immediately expose that. The four together triangulate on the thing that actually matters: shipping value quickly *and* safely, not one at the expense of the other.

The scale of the gap between performance tiers

The most striking finding from this research isn't that elite performers are somewhat better than low performers — it's how large the gap is. Multi-year DORA research has found elite performers achieving roughly 127 times faster recovery from incidents, about 3 times lower change failure rates, and around 208 times more frequent deployments than low performers. These aren't small percentage improvements from tweaking a process. They reflect a fundamentally different way of working — small batches, fast feedback loops, strong automated testing, and psychological safety to respond to incidents well (everything from Modules 2, 4, and 7) — compounding over time into an enormous performance gap.

Why these four, and not something like "story points delivered"

It's worth contrasting DORA's metrics with the kind of metrics many organizations default to instead: lines of code written, number of tickets closed, story points completed, hours logged. Every one of those measures *activity*, not *outcome* — and Module 2's systems thinking already warned about the danger of that: a team can maximize a local activity metric while the actual thing that matters (value delivered to users, safely and quickly) doesn't improve or even gets worse. Lead Time for Changes, by contrast, measures something a customer would actually recognize as valuable: how long from "someone decided to build this" to "a real user has it." Change Failure Rate measures something a customer directly experiences: how often a change makes things worse instead of better. That's the key property that makes DORA's metrics different from most engineering-productivity metrics — they're chosen because of what they've been shown to correlate with, not because they're easy to collect.

A caution for the current moment

DORA's own more recent research adds an important nuance worth knowing: as AI-assisted coding tools have become more common, Deployment Frequency and Lead Time have started to become less reliable signals on their own, because a tool that helps generate code quickly can inflate how fast code ships without necessarily reflecting more real value delivered or being paired with proportionally careful review. This is exactly Module 3's Measurement lesson playing out in real time with a new variable: the four metrics are still useful, but like any metric, they need to be read in context, alongside genuine judgment about whether speed is translating into real outcomes — not treated as a number to maximize in isolation.