IT Projects: 400% Over-Budget and only 25% of Benefits Realized

I personally believe that the root of the problem is called out in the article but in a subtle way (probably in order to avoid offending their readers). The issue is that in most non-software companies, information technology is treated like something separate from the business. The conventional wisdom of a decade ago dominates. That is, if your company is not producing software for sale, it shouldn't be building software. It should be purchased, like Word or Excel.

This line of reasoning seems pretty bullet-proof until you dig into the details and it leads to a number of problems including the issues in the article.

First, lets be clear, no company should seek to build software for internal use that can be purchased at reasonable cost and can can meet their needs without customization. Secondly, companies should seek to purchase or other wise acquire software that implements the generic functions and allows for powerful customizations.

The issue arises when companies purchase something like SAP assuming that their needs are sufficiently supported by the software and that anything they need that is special will be supported by 'configuration'. What then happens is that the company realizes (after investing large amounts of money) that they can either make their business completely generic and lose all strategic advantage over competitors or they can pile on a lot more money to customize the hell out of it. Given the options, most high-level execs prefer more investment over losing strategic advantage. Some companies (I suppose) choose the other option and a lot of IT people think this is a good idea (it isn't.) You might be more likely to succeed at the project but the company will often be ruined in the process.

The error really is taking on a project based on false assumptions. Once that mistake is made, there are no good options. Companies need to stop thinking that information technology is separate from their core functions. Most information technology is only worth having if it is tightly coupled to the design of the business and often technology imposes fundamental constraints on business.

No project management methodologies or helpful tips will address this problem. Companies need to embrace technology and make IT a stakeholder at the highest levels of the organization for the kinds of problems described in the article to be fully addressed.

I take issue with this article for two reasons:

1) Seriously, who does large IT projects anymore? Did they not get the memo that they are, at best, doomed to be way over budget and behind schedule? At worst, they are an abject and complete failure. Hell, the article starts off with a story of Levi Strauss bringing in Deloitte consultants. Even without knowing what they were working on, I could have told you it would fail right there.

2) Black Swans are unpredictable. What's happening on these projects could be seen from a mile away. I guarantee you that any competent developer with 10+ years experience under their belt could have told you these would fail long before they did (and often, before they even started).

The take away shouldn't be that large projects are risky. It should be that you just don't do them in the first place. They have so many problems with them that aren't even technical that it's pointless to even begin them. I've seen companies that always execute small projects well try a big project and have it be a huge failure.

I don't see how the 7 bullet points listed are the hallmarks of agile any more than they are the hallmarks of heavyweight processes.

If 92% of these projects are government projects and many of them are replatforming projects then this study doesn't have much to say about software development, which is agile's domain.

I agree about the Black Swan thing. It would be interesting to get a description of all the projects without providing the final result and get people to predict what they think would happen.

Dean, you are right that the sample list might seem skewed but this is what they follow it up with (from the article), which makes it relevant for all of us

Our sample drew heavily on public agencies (92%) and U.S.-based projects (83%), but we found little difference between them and projects at the government agencies, private companies, and European organizations that made up the rest of our sample.

True Black Swans are unpredictable however in the hindsight, they do not seem improbable. Like they described the Nike project ..

A $5 million project that leads to an almost $200 million loss is a classic “black swan.” The term was coined by our colleague Nassim Nicholas Taleb to describe high-impact events that are rare and unpredictable but in retrospect seem not so improbable.

Why is a project failure if it doesn't live up to the initial plan? It's more likely that the initial plans were wrong. Why do we assume we can predict such complex projects with so many unknowns? Please pass me the crystal ball.

Correction - The article mentions Levis and not Nike.

True Black Swans are unpredictable however in the hindsight, they do not seem improbable. Like they described the Nike project ..

Very interesting stuff. Seems large-scale IT is in even worse shape than indicated by the past studies people have cited over the years. Unclear why the item is categorized under the heading, "agile," though. I see no connection.

I got the hbr article, read it and did some google queries. It turns out that Flyvbjerg is a consultant that supports the "reference class forecasting" position of his firm. His firm specializes in a lot of Civil and Construction projects. Nothing wrong with that of course, but reference class forecasting( finding projects like yours and then basing your budgeting on it) is a challenge. Since every company is different, and since software is so behavioral, I doubt that this forecasting method would do much for us. Remember, this is a complex system, and therefore response better to adaptive than control based process models (google Stacey Diagram: Zone of complexity). The whole concept of the Black Swan that they throw around in the article is about the impossibility of prediction.

The background of the fellows in this article explains why the 7 key steps don't resonant completely with me and others on this list. I've been on "stick to the schedule - limit scope" projects, and they were called waterfall. They release something ontime that no one wants that people who use it haven't seen, and who are usually less productive afterwards. In the article he talks about how Emirates went "Big Bang" and how great that was. Call me skeptical. If Scrum is right, then we use adaptive processes for those things, and we learn and adapt towards deployment.

However, the last 5 are very good, especially #7, focusing on a single target. Focus is what is missing from my enterprises that have had tricky large scale projects.

Could Agile help?

There is clarification on # 2 on the blog

Number two may strike us as an Agile faux pas, not allowing change in scope, but I believe the authors are referring to “Mission Creep” rather than changing featues in a requirements spec. Drifting missions or just unclear goals strikes me as a hallmark of any unsuccessful endeavor.