Creating Better Metrics
[William Playfair] was the first in a series of economists, statisticians and social reformers who wanted to use data not only to inform but also to persuade and even campaign—and who understood that when the eye comprehends, the heart often follows.The article also references the work of Edward Tufte, who has been enormously influential in the field of statistical graphics. In his book Beautiful Evidence, Tufte lays out six "fundamental principles of analytical design":
Show comparisonsKnowing the principles behind good graphics, can agile practitioners use these principles to more effectively present metrics? Better yet, can the principles be used to help identify and create more meaningful metrics? In chapter eight of The Visual Display of Quantitative Information, Tufte applies his design principles to identify patterns of good graphics; one of these is the "small multiple" - showing a large number of similar images next to one another to enable comparisons. A useful application of this pattern shows end-of-sprint burn-up charts for multiple scrum teams, all working together on a single product:
The fundamental analytical act in statistical reasoning is to answer the question “Compared with what?”
Show multiple variables
The world we seek to understand is profoundly multivariate.
Completely integrate words, numbers, images, diagrams.
Document the evidence
Provide a detailed title, indicate the authors and sponsors, document the data sources, show complete measurement scales, point out relevant issues.
Content Counts Most of All
Analytical presentations ultimately stand or fall depending on the quality, relevance, and integrity of their content.
Assuming that these teams have roughly the same sizes, sprint schedules, and assigned story units for each sprint, then this graphic reveals a wealth of insights: unsurprisingly, all the teams struggled during the first sprint; Team A has consistent velocity, but consistently over-commits, requiring stories to be moved to later sprints; Team B has inconsistent velocity; Team C had some troubles early on, but now appears to be on track. Evaluating the performance of a team is often an difficult task for managers, but such apples-to-apples comparisons can help leaders within an organization ask and find answers to the all-important question: "Compared with what?"
Of course, metrics are simply a tool, and - like any tool - can be misused. But the innovative application of graphical design principles can help agile teams create better metrics which, when taken with a grain of salt, can enable agile leaders to find and fix problems within their project teams.
Quite some assumptions
Furthermore, aren't you afraid for suboptimization? The business value (= the product) is delivered by all three teams combined. Suboptimization at team level might not lead to the desired result from a product owner's perspective.
Re: Quite some assumptions
1) I'm currently working with a group of teams on a large product, and in this case all the assumptions are actually valid: roughly the same team sizes working in functional silos that have roughly the same level of complexity. Of course, comparing teams in this way can give insights that are not always obvious. If Team C has a bad sprint, was there a problem across *all* the teams? Does one team seem to consistently have a problem (e.g., not finishing stories) that isn't a problem for the other teams? Charts like this can help identify issues, but they should be the start - not the end - of an investigation.
2) This is just one example of a "small multiple" chart; my real intention here was just to show how Tufte's work can be applied to invent new, more meaningful metrics for software development teams. I would *love* for other readers to post examples of metrics that they've created which go beyond the simple burndown chart.
Srini Penchikala Aug 21, 2014