Studies that Validate Agile and Lean Methodologies

Posted by Dan Puckett on Aug 04, 2010

Sections

Enterprise Architecture,

Process & Practices,

Architecture & Design

Topics

Adopting Agile ,

Methodologies ,

Agile in the Enterprise ,

Stories & Case Studies ,

Lean ,

Agile ,

Business ,

Programming

Share |

RELATED Research

Ralph Jocham asked,

I am searching for some references that show that Agile projects have a higher chance of success than other approaches.

The references should be quotable in a government document ie. come from a trustworthy source.

Studies are a powerful way to convince skeptics of the value in Agile and Lean methods. Conversely, the lack of strong, documented evidence of success can drastically weaken a case for adoption of Agile and Lean methods within an organization. Luckily, there are some studies out there, summarized in this article. 

An extraordinary list of references to Agile studies, XP studies, and related material can be found on the reference list of the book Succeeding With Agile by Mike Cohn.

A highlight of the Succeeding with Agile reference page is a superb and concise set of results at Agile Adoption Rate Survey Results: February 2008. This survey results page is on Scott W. Ambler's Agile surveys site, which contains a wealth of information relevant to evaluating Agile versus Waterfall and other methodologies, and evaluating specific Agile techniques against each other.

Many studies of Agile effectiveness can be found on a page called, unsurprisingly enough, "Studies of Agile Effectiveness." This page contains links to case studies that illustrate both the positive and negative aspects of Agile software development.

The Studies of Agile Effectiveness page is hosted by George Dinwiddie on his Agile Bibliography wiki. The wiki is modifiable by the public, and Dinwiddie has requested that anyone who has links to new studies to add them to the page.

An article by Michael Mah and Mike Lunt entitled, "How Agile Projects Measure Up, and What This Means to You" directly addresses the measurement of Agile effectiveness. From the abstract:

In this Executive Report, Michael Mah and Mike Lunt share observations about real-world agile projects and how they measure up against waterfall and other projects in terms of productivity, schedule, and quality. This was made possible by juxtaposing them against trends from a contemporary worldwide database of more than 7,500 completed projects. Specifically, we look at more than 20 agile releases from five companies. Two of the companies are examined in specific detail, each having achieved best-in-class levels of performance by implementing agile in very different ways.

For information about the effectiveness of Lean, it's hard to beat the books that popularized Lean in the English-speaking world: The Machine That Changed the World, and Lean Thinking. These books have numerous case studies and lots of data about the success of Lean techniques within the manufacturing world.

Lean software development is in many ways different from Lean manufacturing, however. Case studies that are about Lean software development rather than about Lean manufacturing can be found in the paper "Lean Software Development: Two Case Studies" by Peter Middleton.

One disadvantage of many of these studies is that they lack peer review, leaving them open to the criticism that they lack the intellectual rigor necessary to prove the effectiveness of Agile techniques over Waterfall or other methods. Mark Woyna makes the point, however:

Does anyone have a link to peer reviewed studies that prove that Waterfall projects are more successful than process X, or that RUP projects are more successful than Waterfall projects?

Anyone needing that level of "proof" is simply looking for an excuse not to try agile.

RelatedVendorContent

Five Key Practices to Agile ALM

Agile Development: A Manager's Roadmap for Success

Branching & Merging Efficiently: A Guide to Using Process-Based Promotional Patterns

Kanban for Agile Teams

The Agile Tester

Related Sponsor

In today’s hyper-competitive world, later may be too late to adopt Agile development and this Roadmap for Success will help you get started. Download "Agile Development: A Manager's Roadmap for Success" now!

1. Back to top

   Agile isn't a silver bullet.

   by 高 翌翔

   Obviously, Agile isn't a silver bullet.
   
   But the success of Agile development always belongs to people who have the courage to practise Agile Manifesto.
   
   Time will prove everything :)

   Reply

2. Back to top

   The wrong question

   by Michael Studman

   I am searching for some references that show that Agile projects have a higher chance of success than other approaches.

   
   
   It's hard to draw conclusions from such a short message but that seems to be the wrong question to ask if one wishes to avoid confirmation bias. The better question to ask is: what evidence is there for or against Agile as a superior software methodology (which your article then goes on to answer).
   
   

   Anyone needing that level of "proof" is simply looking for an excuse not to try agile.

   
   
   Now that's refreshingly honest.

   Reply

3. Back to top

   Re: Agile isn't a silver bullet.

   by Michael Studman

   But the success of Agile development always belongs to people who have the courage to practise Agile Manifesto.
   
   Time will prove everything :)

   
   
   That sounds like a religious argument. In fact the argument reminded me of a video I watched last night with that Norwegian reporter who spent a month embedded with some Taliban guerillas. Their leader was quoted as saying "During the Russian invasion, someone asked me when the victory will come. The answer was: If the holy warriors are honest and fight only for the sake of God, then victory will come soon. If not, it will take more time."
   
   I don't wish to compare anyone using Agile to the Taliban, of course (that would include myself)! Nor do I intend any disrespect to those who have fought in Afghanistan. I just found the parallels in the two arguments as interesting and illustrative of how the argument by 翌翔 高 is more religious than anything.

   Reply

4. Back to top

   Re: Thank you for your reply :)

   by 高 翌翔

   I think you misunderstand what I mean. For a traditional development team, Agile is a great transformation of processes and people's minds. So you need enough courage to practise it. Of course, you also need to prepare enough before you decide to implement it.
   
   I think the best way to evaluate agile is to use your own mind to judge and practise it.
   
   Time will prove everything :B

   Reply