Creating Better Metrics

A recent article in The Economist pays tribute to three of the finest graphics from the last two centuries: Florence Nightingale's graphic displaying the effect of disease during the Crimean War; Charles Joseph Minard's map showing Napoleon's disastrous invasion of Russia, and William Playfair's chart comparing the price of wheat to the "weekly wages of a good mechanic" over 250 years. It was Playfair who essentially invented the use of graphics to display statistical information:

[William Playfair] was the first in a series of economists, statisticians and social reformers who wanted to use data not only to inform but also to persuade and even campaign—and who understood that when the eye comprehends, the heart often follows.

The article also references the work of Edward Tufte, who has been enormously influential in the field of statistical graphics. In his book Beautiful Evidence, Tufte lays out six "fundamental principles of analytical design":

Show comparisons
The fundamental analytical act in statistical reasoning is to answer the question “Compared with what?”

Show causality

Show multiple variables
The world we seek to understand is profoundly multivariate.

Integrate evidence
Completely integrate words, numbers, images, diagrams.

Document the evidence
Provide a detailed title, indicate the authors and sponsors, document the data sources, show complete measurement scales, point out relevant issues.

Content Counts Most of All
Analytical presentations ultimately stand or fall depending on the quality, relevance, and integrity of their content.

Knowing the principles behind good graphics, can agile practitioners use these principles to more effectively present metrics? Better yet, can the principles be used to help identify and create more meaningful metrics? In chapter eight of The Visual Display of Quantitative Information, Tufte applies his design principles to identify patterns of good graphics; one of these is the "small multiple" - showing a large number of similar images next to one another to enable comparisons. A useful application of this pattern shows end-of-sprint burn-up charts for multiple scrum teams, all working together on a single product:




Assuming that these teams have roughly the same sizes, sprint schedules, and assigned story units for each sprint, then this graphic reveals a wealth of insights: unsurprisingly, all the teams struggled during the first sprint; Team A has consistent velocity, but consistently over-commits, requiring stories to be moved to later sprints;  Team B has inconsistent velocity; Team C had some troubles early on, but now appears to be on track. Evaluating the performance of a team is often an difficult task for managers, but such apples-to-apples comparisons can help leaders within an organization ask and find answers to the all-important question: "Compared with what?"

Of course, metrics are simply a tool, and - like any tool - can be misused. But the innovative application of graphical design principles can help agile teams create better metrics which, when taken with a grain of salt, can enable agile leaders to find and fix problems within their project teams.