How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, GitHub, Google)

Episode Summary

Executive Summary: Nicole Forsgren argues that engineering productivity and developer experience improve when teams measure the right things, define the problem precisely, and optimize for smaller, faster, safer changes. She explains DORA for delivery performance, SPACE for broader complex-work measurement, and emphasizes combining system and people data to guide improvement, not just benchmarking.

Main Topics: Developer productivity vs. developer experience vs. DevOps: Forsgren distinguishes productivity (how much gets done), developer experience (the friction and predictability of writing software), and DevOps (the capabilities/practices enabling faster, safer delivery). She frames them as related but not interchangeable. DORA framework and elite delivery benchmarks: She explains DORA’s four core metrics—lead time, deployment frequency, MTTR, and change fail rate—and shows how they diagnose delivery performance and compare against industry benchmarks. Why speed and stability improve together: A central argument is that shipping smaller changes more frequently reduces blast radius, simplifies debugging, and improves reliability—contrary to the old belief that slowing change improves stability. SPACE framework for measuring complex creative work: SPACE is presented as a broader measurement framework using satisfaction/well-being, performance, activity, communication/collaboration, and efficiency/flow. It helps teams choose balanced metrics for the specific problem they want to solve. Measurement strategy: people data + systems data: Forsgren argues surveys/interviews and telemetry are complementary. System data can show what happens; people data can reveal heroics, friction, and gaps systems miss. She recommends starting with people data and adding system instrumentation over time. Common pitfalls and change management: Teams often fail by not defining the problem crisply, using metrics as vanity targets, or trying to roll out change without both top-down executive support and bottom-up engineer buy-in. AI’s impact on engineering productivity: AI tools change the nature of work by shifting effort from writing code to reviewing it, altering cognitive load, learning, trust, and reliance. Forsgren says AI can boost capability, but not in the simplistic 'halve headcount' way.

Key Arguments: Teams must start by defining the exact problem or goal; otherwise different stakeholders optimize different things and create conflicting workstreams. Developer productivity is best understood holistically, including community, well-being, sustainability, and burnout reduction—not just output volume. Faster delivery can improve quality and stability because smaller, more frequent changes are easier to test, debug, and recover from. DORA metrics are useful not just for benchmarking but for identifying constraints and deciding the next improvement step. SPACE exists because complex work needs balanced metrics across multiple dimensions; no single number (like LOC or commits) is sufficient. People data and system data should be triangulated because they answer different questions and can contradict each other in valuable ways. Teams should avoid starting from data alone; instead, start with words/hypotheses, then map them to metrics and validate the relationship. AI changes developer workflows and may require new metrics, potentially adding dimensions like trust or reliability. Improvement initiatives succeed when they are communicated in the audience’s language and tied to business value and developer pain points. Decision-making becomes clearer when criteria are explicit, weighted, and scored; often the process reveals the answer itself.

Data Points: Teams with uncertainty around goal definition: 80% - Forsgren says most people she works with, even executives, struggle most with defining the exact problem or goal. DORA elite deployment frequency: On demand - Elite performers can deploy whenever needed. DORA elite lead time for changes: Less than a day - From code committed to code running in production/ring zero. DORA elite time to restore: Less than an hour - Mean time to restore service after an incident. DORA elite change fail rate: 0% to 15% - Percentage of changes that cause incidents or require human intervention. DORA high lead time for changes: 1 day to 1 week - The next benchmark tier below elite performance. SPACE dimensions: 5 - Satisfaction & well-being, performance, activity, communication & collaboration, efficiency & flow. Recommended minimum SPACE dimensions to use: 3 - Forsgren recommends using at least three dimensions together to maintain balance and avoid optimizing one metric at the expense of others. AI workflow shift: ~50% of time spent reviewing vs. writing - She cites an MSR paper using the CUPS model showing code review taking about half of time with AI-enabled tooling. Book-timing estimate: About 1 year - She expects the new measurement book to take roughly a year to complete.

Pivotal Quotes: "when you move faster, you are also more stable" — Nicole Forsgren: Explaining the DORA finding that speed and stability move together rather than trading off. "DevOps is not a tool chain you buy" — Nicole Forsgren: Clarifying that DevOps is a set of technical, architectural, and management capabilities, not just marketing-label software. "start with words. Do not start with data" — Nicole Forsgren: Describing her four-box framework for defining a hypothesis before choosing metrics.

Implications: For engineering leaders, the path to better productivity is clearer problem definition, balanced measurement, and smaller, more frequent delivery. For the industry, AI will amplify the need for rigorous measurement frameworks that capture speed, quality, trust, and human experience.