IT Projects: 400% Over-Budget and only 25% of Benefits Realized
An alarming study by Flyvbjerg and Budzier published in the Harvard Business Review has made everyone stand-up and take notice. The coherent advice being that IT projects are much more riskier than we think.
The study showed that amongst the 1400+ projects surveyed, on an average 27% were over budget. 16% of the projects could easily be categorized as Black Swans. They had a cost overrun of over 200% and were late by almost 70%. These were the projects which could cause stunning failures and bring down companies.
Boris Gloger mentioned that the results were frightening.
These results are significant. But even more frightening are the conclusions of the authors of Harvard Business Review article. They write: If you want to avoid the slow death caused by IT projects you must be prepared not only to spend 400% more than planned on the project, but to reap only 25% of the expected benefits. If you keep this in mind you can possibly prevent a company from being killed by an IT project.
Paul Glen mentioned that one would expect the distribution of project overruns to look like a bell curve, but surprisingly that is not the case. A disproportionate number of projects were huge failures which had the potential to destroy complete organizations.
Ed added that it was amazing to notice that in the last 15 years the industry had not got any better at delivering large IT projects. One of the possible reasons for the results to look the way they look could be that the study pointed to a lot of projects where re-platforming was involved.
My theory is that it is these very large re-platforming projects that are frequently the issue. They touch so many systems, so many discrete business units involved across the globe, and the desire to do customization so they become impossible to upgrade.
Ed mentioned a similar re-platforming project which was successful as it had all the ingredients of a good Agile project.
An example from the article of a successful project by the Emirates bank in 2006, shows a successful way to run a large complex project. The lessons learned from this successful project were to:
1. Stick to schedule
2. Resist changes to the projects scope
3. Break the project into discrete modules
4. Assemble the right team
5. Prevent turnover among team members
6. Frame the initiative as a business endeavor, not a technical one
7. Focus on a single target – readiness to go live, measuring every activity against it.
Boris added that another reason for such disappointing numbers could be that we are addressing wrong people.
But maybe we are addressing the wrong people , like Ken Schwaber says in his new, soon to be published book. Possibly IT customers, software development, and product development need to simply not accept any more that IT departments, QA departments and process people can not create finished, tangible and measurable results every two weeks.
Mary-Ann Massad mentioned that a good governance model could have saved the day for many organizations. According to Mary-Ann,
Real governance means leadership. Real governance means courage-the courage to tell the truth even when it will not be well-received. Too many "well"-run organizations operate under a culture of fear and "group-think". This is why IT projects or any projects for that matter, have the ability to bring an organization to its knees. We have to work on building organizational cultures that welcome honesty, diverse opinions, and integrative thinking.
Thus, one could dive deeper to analyze the shortcomings and learn from them, however the numbers do suggest a serious re-think in the way projects are executed.
This line of reasoning seems pretty bullet-proof until you dig into the details and it leads to a number of problems including the issues in the article.
First, lets be clear, no company should seek to build software for internal use that can be purchased at reasonable cost and can can meet their needs without customization. Secondly, companies should seek to purchase or other wise acquire software that implements the generic functions and allows for powerful customizations.
The issue arises when companies purchase something like SAP assuming that their needs are sufficiently supported by the software and that anything they need that is special will be supported by 'configuration'. What then happens is that the company realizes (after investing large amounts of money) that they can either make their business completely generic and lose all strategic advantage over competitors or they can pile on a lot more money to customize the hell out of it. Given the options, most high-level execs prefer more investment over losing strategic advantage. Some companies (I suppose) choose the other option and a lot of IT people think this is a good idea (it isn't.) You might be more likely to succeed at the project but the company will often be ruined in the process.
The error really is taking on a project based on false assumptions. Once that mistake is made, there are no good options. Companies need to stop thinking that information technology is separate from their core functions. Most information technology is only worth having if it is tightly coupled to the design of the business and often technology imposes fundamental constraints on business.
No project management methodologies or helpful tips will address this problem. Companies need to embrace technology and make IT a stakeholder at the highest levels of the organization for the kinds of problems described in the article to be fully addressed.
1) Seriously, who does large IT projects anymore? Did they not get the memo that they are, at best, doomed to be way over budget and behind schedule? At worst, they are an abject and complete failure. Hell, the article starts off with a story of Levi Strauss bringing in Deloitte consultants. Even without knowing what they were working on, I could have told you it would fail right there.
2) Black Swans are unpredictable. What's happening on these projects could be seen from a mile away. I guarantee you that any competent developer with 10+ years experience under their belt could have told you these would fail long before they did (and often, before they even started).
The take away shouldn't be that large projects are risky. It should be that you just don't do them in the first place. They have so many problems with them that aren't even technical that it's pointless to even begin them. I've seen companies that always execute small projects well try a big project and have it be a huge failure.
If 92% of these projects are government projects and many of them are replatforming projects then this study doesn't have much to say about software development, which is agile's domain.
Re: Black Swans?
Our sample drew heavily on public agencies (92%) and U.S.-based projects (83%), but we found little difference between them and projects at the government agencies, private companies, and European organizations that made up the rest of our sample.
Re: Black Swans?
A $5 million project that leads to an almost $200 million loss is a classic “black swan.” The term was coined by our colleague Nassim Nicholas Taleb to describe high-impact events that are rare and unpredictable but in retrospect seem not so improbable.
Project failure or planning failure?
Re: Black Swans?
True Black Swans are unpredictable however in the hindsight, they do not seem improbable. Like they described the Nike project ..
I think that Marc's point is that a lot of people could (and probably did) predict these issues long before they occurred. If something occurs as frequently as large project failures do, calling it a 'black swan' doesn't seem inappropriate.
Aside from my comments above, the dynamic of such projects is that middle management cannot openly express doubts for fear of demoralizing the team. It's not that the failures are improbable based on history (quite the opposite) but that the people involved are simply unaware of the risks.
Skeptical of this Article
The background of the fellows in this article explains why the 7 key steps don't resonant completely with me and others on this list. I've been on "stick to the schedule - limit scope" projects, and they were called waterfall. They release something ontime that no one wants that people who use it haven't seen, and who are usually less productive afterwards. In the article he talks about how Emirates went "Big Bang" and how great that was. Call me skeptical. If Scrum is right, then we use adaptive processes for those things, and we learn and adapt towards deployment.
However, the last 5 are very good, especially #7, focusing on a single target. Focus is what is missing from my enterprises that have had tricky large scale projects.
Re: Skeptical of this Article
Number two may strike us as an Agile faux pas, not allowing change in scope, but I believe the authors are referring to “Mission Creep” rather than changing featues in a requirements spec. Drifting missions or just unclear goals strikes me as a hallmark of any unsuccessful endeavor.