When defining a business case for adopting agile, the question can arise how you can measure the business value that can be delivered using agile software development.
In the blog post agile metrics: measuring process value Michael Moore explains why it is important to measure the value delivered by agile:
(…) Agile is not just about efficiency, it’s also focused on effectiveness (value delivery). After all, what good is a process/method that helps you to do what you do faster/cheaper, but ultimately fails to deliver more value to your end users? Agile, therefore, offers the promise to both decrease costs and to increase revenue.
So, if your emphasis, and maybe your sole reason for choosing to go Agile, is to be more efficient, you will likely see the results you expected; your stuff is going out the door faster. But, if this excludes a focus on increasing revenue/value (i.e. product results), then ultimately you’re really just delaying the inevitable death of your organization. No amount of cost reduction can make up for a lack of revenue production.
Michael talks about applying the so called “pirate metrics” to measure the value of agile: acquisition, activation, retention, referral and revenue. When measuring value, initially the focus should be on metrics that provide information about the acquisition of agile. When acquisition has increased the focus can shift to measuring activation:
With this model as an example, if we apply it back to the operational/process side of Agile Transformation, what would we consider to be our metrics? Would the revenue equivalent be velocity (I know, don’t compare!)? Would acquisition be equivalent to number of people trained in a boot camp?
Earlier InfoQ interviewed Capers Jones on measuring for agile adoption. Capers mentioned some basic measurements that can be used when migrating from one methodology to another; these metrics can be used to measure agile value:
For measuring productivity the two most widely used measures in the world are “function points per staff month” and its reciprocal, “work hours per function point.” For the purposes of judging productivity, rates above 10 function points per staff month are good; rates below 7 function points per staff month are not so good. Some Agile projects have topped 15 function points per staff month, and this almost never happens for waterfall.
For measuring quality the most effective measures are “defect potentials “normalized using function points and “defect removal efficiency.” Defect potentials are the sum of bugs that will be found in requirements, design, code, documents and bad fixes. A defect potential above 5 per function point is bad; a defect below 3 per function point is good.
Ken Schwaber wrote the blog post agile value where he discusses how to measure the value created by IT investments:
(…) value is defined as the financial benefit that an organization receives for expenditures. When measured, value can encompass an entire organization, or be constrained, such as to a single division or product line. Regardless, it must encompass those areas affected by the expenditure.
The organizational value of investments can be measured with a single metric called the agility index which is defined in the agility path framework. This metric combines the results from several metrics:
- operational metrics, measuring an organization’s efficiency. These metrics are heavily slanted toward IT, both the development of software and the deployment of it in the organization. They include flow, cycle time, stabilization, quality, utilization rate, and release cycle time. Business operational efficiency is not included in these measurements, but will be addressed in 2014 as part of the Kotter International (John Kotter) Accelerate! program.
- organizational metrics, measuring the impact of that efficiency of investment on the marketplace and the value the organization accrues. These metrics are revenue per employee, customer and employee satisfaction, and product/system cost ratio.
You can measure the value that is delivered at the end of a Scrum sprint, as Ken describes:
In the agile project, each Sprint or iteration is a complete project. It has requirements, budget, and due date. At the end, it has a completed set of software functionality. Based on what is completes, another project may or may not be initiated, adding more functionality to the functionality just completed. Each Sprint is measured on its own.
The Cutter blog post it’s far too easy to measure agile’s value in the wrong way explores the requirements for measuring the value of agile. Tom Grant provides some guidelines on what to calculate and the pitfalls to be avoided:
- “The value of Agile methods is not the same as the value of software that Agile teams produce”: Estimating the business value of stories doesn’t help to warrant the investment of adopting agile in the organization.
- “Business value exists at the level of business activities”: It is not the stories that have business value but the business activities using the delivered software. Tom states that “Value exists at the level of epics and themes”.
- “The portfolio must have some role in calculating value”: The business value of the software that is delivered is also partly related to functionality which is not included.
- “Value is a reciprocal measure”: Measuring value needs to include both the producers and the consumers of the technology.
- “Nobody believes in exact measures, but they’re useful anyway”: Although you can’t exactly calculate the value people still want a figure to discuss it.
- “ROI calculations can be so big as to be unbelievable”: People may be skeptic when they see high figures, but they might be correct.
- “Investment on return may be more important than return on investment”: Not investing enough in agile could result the adoption to fail.
- “There are lots of other measures than ROI”: Other metrics could also be useful in a business case for agile adoption.
Do you measure the business value of agile adoption?