BT

Can Earned Value Leverage Agile Methods?

by David Bulkin on Jun 17, 2011 |

The debate over the value of Earned Value Management (EVM) and integrating it into agile rages heavy as agile penetrates into more large scale IT projects that require EVM. Opinions vary but some believe that not only can agile projects apply EVM; EVM with agile is superior to EVM without agile.

Glen Alleman explains the basics of EVM: "Earned Value measures progress as physical percent complete against the planned percent complete." The Defense Acquisition University (DAU) Gold Card provides a sample visualization of how EVM represents current variance to provide input for future adjustments (note that traditional EVM terms are used).

  • The cost variance shows the difference between the Budgeted Cost for Work Performed (BCWP), often called the Earned Value (EV) using newer terminology, versus the Actual Cost for Work Performed (ACWP). In this example, the project is currently over budget.
  • The schedule variance shows the difference between the Budgeted Cost for Work Scheduled (BCWS) versus the BCWP expressed as dollars to approximate conformance to schedule. In this example the project is currently behind schedule.
     

DAU Gold Card Example

In a May 2011 blog post titled Agile and EVM Strategies, posted on Dr. Dobb’s site, Scott Ambler acknowledges that EVM can be applied to agile projects, but he questions its value for IT development projects, agile or not.

The fundamental problem with EVM is that it has very little to do with earning value and everything to do with managing against what often proves to be naive/fictitious schedules and estimates committed to far too early in the project. Although EVM is interesting project management theory, and I have no doubt that it may work in some non-IT domains, it proves to be a really bad project management strategy in practice for IT development projects.

Although EVM can in fact be applied to agile projects, in my opinion EVM is a questionable practice, regardless of paradigm. Organizations that are trying to govern their IT project teams should monitor them in such a way that accurate and timely information is presented. This is clearly not the case with EVM

Others provide a different perspective.  Glen Alleman wrote a post in response to Scott's statements and, in a private email exchange, Glen explained the symbiotic relationship of EV and agile.

EV and agile have a symbiotic relationship. EV can forecast the Estimate at Completion in units meaningful to the funder – dollars. Agile can adapt to changes in requirements will relative ease, because of the relationship between the customer and the development team (single team). Now add 1,000's of requirements, a complex architecture (ERP to manned space flight guidance), multiple funding sources, a disconnected customer (we’re in Phoenix and the customer is in Houston and we meet once a month), and a myriad of other “complications” and Agile can still provide value at the software development level and EV becomes one of several "governance" process when spending other people's money. Be that money government or shareholders.

Derek Huether, who, until recently, was an advisor to a PMO for the United States Federal Government, further states how agile approaches improve EVM, "If the vendor is forced to deliver value to the customer at the end of short iterations, there is less opportunity for some kind of manipulation of Cost Performance Index (CPI) or Schedule Performance Index (SPI) to be attempted."

Dr. Norm Brown, of the Center for Program Transformation, states the case for how agile improves EVM more strongly, "EVM becomes highly effective when agile techniques are applied to the program."

For those who are contractually obligated to adhere to EVM requirements, this is a practical issue, not a philosophical debate. Fortunately there is free, high quliaty, EVM information on the web, with several resources listed below, allowing you to make an educated decision.

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Additional Resources by John Rusk

David,

Nice to see this discussion of the issues. If you're interested, you can find some additional resources on this topic here www.agilekiwi.com/earnedvalue/resources/ I wrote them as an agilist, who unexpectedly found himself invited to write on Earned Value for the traditional (non-agile) community and, perhaps even more unexpectedly, found the that resulting articles met with approval from both the Earned Value community and the agile community.

Simple Concepts are Highly Effective by Glen Alleman

David, Thanks for the references. One simple idea around Earned Value is that EV "earns," the budget. For projects that spend other peoples money, the budget for the work is a critical management concern. EV answers the question "did I earn my budget." Earned Value = Budgeted Cost x Percent Complete. This number is in units of measure meaningful to the decision makers - money and time.

The ability to measure physical percent complete is a natural outcome of agile projects. But since agile development does not naturally measure progress in money, some form of monetization is needed. This is one role for Earned Value. As well agile projects do not capture actual cost for the progress being made. In many cases, this is simple because the team size and periods of performance are fixed. 4 weeks with the same team, simple team size x 4 weeks x burdened rate. But if the development team is variable in size and the periods of performance overlap with other teams, the funding profile varies. Monetizing the work is then critical to answering the question "how much will this cost when we're done?"

John has provided many resources from the point of view of agile. There are numerous resources from the point of view of EV, where agile is the development method. As suggested by other, Agile + EV = Success in domains and contexts that are beyond a simple linear development, with a fixed team, a local customer, and a funding source willing to execute the project in a time boxed manner, willing to accept the results of the fixed budget.

Re: Additional Resources by David Bulkin

John,

Your agile site has lots of EVM information, making it useful for those interested in this discussion.

In your Earned Value in a Nutshell page, you distill EVM into two simple rules:
1. Assess your past performance by comparing actual cost to actual progress, numerically.
2. Assume your future performance will be similar (unless you take corrective action)

You expand on the second rule stating that the biggest challenge to it is psychological.
“When a project is going badly, we want to believe that the future will be better than the past. We instinctively convince ourselves that we can catch up. Sometimes that’s true (as long as our corrective action is appropriate), but sometimes it’s not. The great thing about EVM is that it provides an objective measure, independent of our tendency to kid ourselves. That objectivity can feel uncomfortable at times — particularly on those occasions when we actually are kidding ourselves :-)”

Building on this point, the addition of agile to EVM can make the analysis even more objective, as the deliverables against which the value is earned are tangible units of functionality. This is the essence of the change in mindset as represented by Derek Huether and Dr. Norm Brown. Not too long ago, many looked at agile as incompatible with large projects requiring EVM, now agile techniques are being viewed as a potential way of making EVM better.

Re: Simple Concepts are Highly Effective by David Bulkin

Glen,

Your comment goes a step further than does the news item, pointing to the potential advantage of adding EVM to agile for complex agile implementations (e.g. instead of adding agile to EVM). Maintaining journalistic integrity, I won’t agree or disagree, but I will state that this is an interesting proposition.

Re: Additional Resources by John Rusk

David,

That's a nice way to phrase it. I've noticed hints of the change in mindset, but haven't yet read Derek Huether and Norm Brown's work. Thanks for pointing them out.

earned variability doesn't provide value for product design (knowledge work by Jeff Anderson

The problem is that Eanred Value assumes that prtially completed work can be tracked for knowledge work. It can't, the only time you have a really accurate status is when you say something is done or not done, almost done doesnt cut it. So the only way you can calculate earned value is if you count the items you have shipped to your customer vs those you haven't.

The other problem is that a major source of variability has nothing to do with how the project is delivered, but the nature of the work. When ever you use technology to build new application the actual value of the work is highly unpredictable, look at marketing forecasts vs actuals if you don't believe me. the actual earned value here is just tracking if you are on plan, but the actual value delivered still fluctuates wildly.

We are better served by tools that allow us to make real time decisions based on our current understandings of schedule, profit, Dev cost, and Maintanance cost. If we ignore anyone of these inputs then we are guaranteed to make sub optimal decisions, earned value does not look at all these factors, this is why schedule and development effort get a distorted amount of attention.

Read anything by Don Renersten, for advice on how to do this.

Re: Simple Concepts are Highly Effective by Glen Alleman

David,

Once the project moves beyond the "linear" processes of taking stories from the backlog, working them in the iteration and having the customer manage the contents of the backlog, then there needs to be so sort of "manager" of the work processes and measurement of the effectiveness of this work.

In the end though it comes done to connecting (and disconnecting) the development of software from the backlog and the management of the resources around the development of the software. This doesn't mean the management of people and all the processes around developing software using Agile. It means managing (governing) the process on behalf of the business value generation.

There is lots of words about "business value," but the actual mechanical processes needed to deliver that "value" aren't usually forth coming. Here's one approach to connecting "business value," with the performance measures of a project using Earned Value, slidesha.re/g0i0as.

When EV is mentioned with Agile, there are three sets of "performance variables," involved:

1. The charts showing the performance of the backlog. But the units of measure are useful for the developers. But not in units for the business - money. The monetization of these stories is not credible to finance people, because the burdened cost of each story is not defined.

2. The BCWS (burdened or wrap rate) cost of the work is the starting point of EV. How much are we budgeting for the development of a feature, story, iteration, or the whole project?

3. The "business value," somehow connected to the strategy for the project. The units of measure here are business specific, build around the Balanced Scorecard for example. There are other IT strategy approaches, but BSC is a good one to apply to IT projects and the business processes enabled by them.

Connecting all three of this is shown on Page 48 and 49 of the referenced briefing.

Re: Additional Resources by David Bulkin

John,

Derek Huether and Dr. Norm Brown gave their quotes directly via phone and email so they don't necessarily have publicly available content available on the web. The interesting aspect is that they are not necessarily unique in their thinking about the advantages of adding agile to EVM.

Re: earned variability doesn't provide value for product design (knowledge by David Bulkin

Jeff,

Since this is a comment (e.g. not a news article), I can take my news hat and state that I am a fan of Reinersten's work. I read "The Principles of Product Development Flow" soon after it was published.

I can also conjecture that most people in the agile community agree with you that work is either done or not, but I would further conjecture that many in the EVM community feel the same way. This is why the 0/100 earning rule exists (e.g. no credit unless 100% complete).

It is also clear that EVM allows you to prioritize and deliver based on business value, thus further addressing the points made in your comment.

Putting this together, if a program requires EVM, and your organization is deciding whether to bid on the work or not, you have a few options, including:

  • Choose not to bid

  • Try to convince the prospect/customer to abandon EVM

  • Adhere to the EVM requirements using traditional software development methods

  • Adhere to the EVM requirements while using agile approaches to software development



The crux of the article is that perhaps we are reaching a juncture where we can acknowledge that the fourth approach, adding agile methods to EVM, is a viable option.

On the flip side, I can understand the consternation that many have about the way that EVM is applied by some contractors. It is easy to subvert the earned value process by continual rebaselining, creation of level of effort tasks (e.g. you get credit for showing up),tracking at a very high level, using something other than a 0/100 earning rule, etc.

So, the question is, can adding agile practices to projects requiring EVM, result in more meaningful EVM?

Re: Simple Concepts are Highly Effective by David Bulkin

Glen,

To summarize what I believe to be your perspective, EVM, and other approaches, like BSC, can provide value when added to agile. This builds on the scope of the original news article and proposes an interesting line of thought for those in the agile community to ponder. Let's see if anyone else can provide their input and experience.

Re: earned variability doesn't provide value for product design (knowledge by Jeff Anderson

I can operate under EVM, and have done so times, if the client requires it than it is often easier to go with the flow than tackle it head on.

I also think you can do it in a agile context, certainly the all or nothing approach would be the ideal way. The question in my mind is what value does it bring, and isthe economic cost of EVM worth the economic benefit?

I don't buy into calculating the value side of EVM using projected value, actual value of a feature can vary by 3-5x from projected, so EVM can't really be calculated unless you well after you have deployed something and are earning revenue.

I look at EVM as a way to fudge economic benefit for long running projects, if you can get the number quickly and easily than the fudge will have value, in most cases this has not been the case, to much effort to get a number that is not very accurate.

If there is a cheap way to get EVM , than I say go for it, it can inform decision makers. Just don't confuse it with reality.

Re: earned variability doesn't provide value for product design (knowledge by Jeff Anderson

Oh and Dave great to see you are familiar with Don's work, and wanted to say you present a very balance and thoughtful perspEctive..

Re: earned variability doesn't provide value for product design (knowledge by David Bulkin

Jeff,

Thanks for the kinds words about balance. InfoQ provides a more balanced perspective than does a typical blog, which is what makes it such a great forum.

In respect to your comments about the cost of EVM versus the value, as agile gains a foothold in large scale projects requiring EVM, we can speculate that the effort required for EVM reporting will diminish as the quality of tool support grows. I expect that one of the most time consuming aspects is / will be the up front creation of a deliverable oriented WBS that spans the entire project. Most agile teams don't look forward for the whole duration of their project, and most don't specify the anticipated costs in dollars.

Converting story points estimates to dollar estimates can be achieved in a number of lightweight ways, so I speculate that the largest cost will be the additional up front planning. As with everything else, this additional up front planning will have some benefits in addition to the costs.

Re: earned variability doesn't provide value for product design (knowledge by Glen Alleman

Jeff, You're confusing business value with earned value. The "value" earned in earned value is the budget for that work. That budget is "fixed" at the beginning of the iteration. It is NOT variable. This seems to be a continuous confusion starting with Scott and going most places in the agile community.

Re: earned variability doesn't provide value for product design (knowledge by Glen Alleman

Jeff, again the "value" in EV is NOT the "value" you speak of. So it sounds like the EV projects you've been are not compliant with ANSI-748B.

Re: earned variability doesn't provide value for product design (knowledge by Glen Alleman

David,

You have identified one problem. But this problem - no definitive WBS for the planned work - is bot a strength and a weakness of agile and all other project methods. If we don't know what done looks like, then we can't measure progress toward done other than the consumption of money and the passage of time. If the customer is happy with that, then no one cares.

But if the customer has finite funds and a fixed amount of time, and wants specific features to show up inside that boundary, then someone has to create a description of what "done" looks like and connect the dots between the development of the parts in the "done" with the work effort to produce them. This can partially be defined in the WBS and is fully defined in some for of top level flow of the deliverables.

Re: Simple Concepts are Highly Effective by Glen Alleman

David,

I'd conjecture the inverse. Agile adds value to those items. BSC states "why," the Integrated Master Plan says what order the top level deliverables are produced. It's these top level deliverables that produce the "business value," for the customer. Here's a post around the structure of that "business value" map bit.ly/kOfdyx.

Re: earned variability doesn't provide value for product design (knowledge by Glen Alleman

Jeff,

Regarding the "cost" of EV. In projects larger than developers and customer in the same room working on emerging requirements through iteration until the money runs out, there are 11 core practices for program management. Page 14 of slidesha.re/mq37vD lists them. If you're not in this domain, don't do EV to start with. It's a none issue. But if your in a domain that spends other peoples money and has fixed budget and period of performance for delivery and a minimum set of needed features, then agile alone won't get you there because it doesn't produce information directly for these 11 mandatory project management processes.

Re: earned variability doesn't provide value for product design (knowledge by Glen Alleman

David,

The answer to your last question is a resounding YES. Agile development - fixed iterations, 100% working products - is the perfect fit for the work package rolling waves of EV. BUT if you're program doesn't require EV, don't consider it until you've come the grips with wanting to manage the project using the 11 processes of good project management shown on page 14 of slidesha.re/mq37vD. If your project isn't interested in those processes, then don't connect agile and EV.

Re: Simple Concepts are Highly Effective by Glen Alleman

David,

I'd invert the statement. Agile adds value to EV and BSC. If your program doesn't need EV (not required by contract) or your organization is not prepared to adopt the EV paradigm of performance management, AND the projects are performing acceptably, within time and cost constraints, then EV added little value. If the org is not ready for EV, then it is likely to fail and result in the observations made by Scott Ambler. But that has little to do with the effectiveness of EV, and nearly everything to do with the readiness of the organization. This is the case for IT organizations to Joint Strike.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

20 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2013 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT