BT

James Shore: The Decline and Fall of Agile

by Chris Sims on Nov 17, 2008 |

James Shore has declared agile to be in decline. He cites the many teams doing 'sprints' and stand-up meetings, without adopting any of the technical practices necessary to produce high-quality software over the long-haul. In his estimation, this has led to thousands of Scrum teams doing agile so poorly that they will almost certainly fail, and possibly take the agile movement with them.

James lays a large portion of the blame on Scrum, and the misapplication of Scrum. He compares Scrum with Extreme Programming (XP) and notes that Scrum intentionally leaves out the engineering practices that XP includes. Scrum is silent on topics such as pair programming, test driven development, continuous integration, and test automation. Without such practices, a team can quickly build a large, buggy, and unmaintainable code base. This then becomes a weight around the neck of the team, preventing them from responding quickly to change, as an agile team should.

James thinks it's not all the fault of Scrum, however, as each team must take responsible for its own success or failure. Many are choosing to adopt only the superficial, and easy, parts of Scrum such as short development sprints and daily stand-up meetings, while ignoring harder, yet critical practices such as reflecting and improving. Via this process, teams are empowered to identify and adopt the engineering practices that they need to deliver shippable software every iteration. Unfortunately, many teams fail to take this step.

Several commenters preferred the view that the problem isn't Scrum itself, but the people who are implementing it poorly. For example, Dustin Whitney said "To me you are just describing mediocrity​, which will never go away. I don't think it's fair to blame scrum for the failures of mediocre developers and project managers."

In James' view, the failures, regardless of reason, may lead to agile being labeled a fad, and fading away.

So, unfortunately, a lot of self-described Agile projects are going to fail. They're failing right now. And eventually Agile will take the blame, and it will pass, as all fads eventually do.

Simon Kirk responded to all of this more optimistically:

I don't disagree with the premise that a lot of what's being done now under the name of "agile" is anything but. On the the other hand I truly think that this stage is an inevitable step along the road of a wider adoption of agile (by which I mean properly done agile, by the way).

Is agile a fad? Is it too hard for most teams to do effectively? Or is agile just experiencing growing pains on the way to ever-wider and more successful adoption? Leave a comment and share your views.

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Gartner hype cycle by franck villaume

Is there any link between Agile decline and the gartner hype cycle ?
en.wikipedia.org/wiki/Hype_cycle
Are we in the "trough of disillusionment" phase ?

Yet another clueless manager by Ilya Sterin

First, methodologies are only as good as the people that apply them. I hate the thought of process over people. Intelligent people will find a way to produce good software, agile or not. Morons will fail even with the process. Software development is more art than it is science, though I wish the people that never made it as software developers would stop trying to pile process on top of process and think that engineers are code monkeys that can develop good software by following some process. Process is good, but smart people are better.

Also, it's not that agile is failing, software projects are failing and have been failing before agile and will be failing after. Again, this is art and creativity is required not process.

Not a condemnation of Scrum by James Shore

Although a lot of people have taken my essay to be a condemnation of Scrum, that wasn't my intent. Instead, I was trying to highlight the failures that I see and the factors that contribute to those failures. The biggest problem isn't Scrum or even CSM training, it's people eating dessert and skipping their vegetables.

Re: Yet another clueless manager by Dave Rooney

Also, it's not that agile is failing, software projects are failing and have been failing before agile and will be failing after. Again, this is art and creativity is required not process.

That may be true, but perception is reality. There is a backlash against agile that exists for a variety of reasons. There's even a backlash within the agile community - some against Scrum, some against XP1E, some against XP 2E, etc.

As a manager looking to improve your teams, are you going to incur the risk of trying something that may not be around in a year or two?

There needs to be a consistent, cohesive message about what this thing called "agile" is and is not. As a community, we need to be honest that no process under the agile umbrella is a quick fix - they all require consistent application and reflection, and they likely all require some form of facilitation from an experienced practitioner.

Dave Rooney
Mayford Technologies

Mainstream kills by Hermann Schmidt

Once the mainstream has taken possession of a term or concept, it is almost guaranteed to be burned. It gets worse with the amount of money the industry expects to earn with it.

The mainstream consists of individuals who are semi-educated and therefore not able to understand the full concepts. They work by copying templates.

The ones who truly understand it create the templates. The others follow by example with mixed success.

Take object orientation, webservices or SOA as prominent examples. Burned or about to be burned. Misused or abused as labels where they don't fit. SOA is the current super burner. It's already beginning to emit smoking signs.

From all new concepts, something useful has been or will be left over, which will be carried further and actually improve things. It's never been the full package. I think Agile will go through the same mangle. There is no escape from the mainstream.

Uncle Bobs Response by Sebastian Kübeck

...to that and many similar postings: Dirty Rotten ScrumDrels

However, I could not more agree with this one:


It makes no sense to blame Scrum for the fact that we don’t write tests and don’t keep our code clean. We can’t blame scrum for technical debt. Technical debt was around long before there was scrum, and it’ll be around long after. No it’s not scrum that’s to blame. The culprits remain the same as they ever were: us. Of course it’s true that a two day certification course is neither necessary nor sufficient to create a good software leader. It’s also true that the certificate you get for attending a CSM course is good for little more than showing that you paid to attend a two day CSM course. It’s also true that scrum leaves a lot to be desired when it comes to engineering practices. But it is neither the purpose of scrum nor of CSMs to make engineers out of us, or to instill the disciplines of craftsmanship within us. That’s our job!

Unfortunately true by Stephan Oudmaijer

I see this happening in my current project team aswell. If you want to do Agile, just using the bits that suits you most aint enough. IMO the team (and customer) should also be a little bit Agile aswell.

Re: Yet another clueless manager by Jason Yip

"A bad system will beat a good person every time." -- W. Edwards Deming

I've met plenty of intelligent people, who other people think are morons, that were systematically beaten down by a dysfunctional system.

Celebration of heroics and the talent myth along with focusing purely on results with no consideration of how they were achieved... well that's part of the dysfunction.

Agile has to be XP and Scrum together, one without other leads to failures by Sachin Mehra

It is not the decline and fall of Agile, but of those who are unable to understand the methodology. Those people practice what they believe to be Agile and when they fail, they blame Agile for it. I have failed with Agile several times, however I learned something new every time. The only learning I pass on is that you cannot go by the book, that’s what Agile is all about. Also, Agile has to be XP and Scrum together, one without other leads to failures.

Even good things fail when they are hyped by Hans Hartmann

I am from Austria and in contact with quite a lot of software companies. Recently, I am very often confronted with a statement like "we are doing scrum, now!"
The courses in scrum are expensive and about as effective as if somebody went to a pianist and tried to learn to play the piano in 5 days.
Apart from doing to right things, utilizing a method like scrum requires some additional details:
1) the overall architecture must be understood
2) the scrum-master must possess a very vital and apprehensive intelligence
3) Test does not only consist of the tdd-part, that will basically function for unit- and component test, but has to be added later on with just the regular know-how concerned with test.
If either one is lacking, the scrum project will fail, I would daresay.
Only lately, I found a manager who told me, that they had attempted to do scrum in two projects both of which failed. At least he admitted, that they were obviously leaving out some vital point.

Re: Agile has to be XP and Scrum together, one without other leads to failu by Dave Rooney

It is not the decline and fall of Agile, but of those who are unable to understand the methodology. Those people practice what they believe to be Agile and when they fail, they blame Agile for it. I have failed with Agile several times, however I learned something new every time. The only learning I pass on is that you cannot go by the book, that’s what Agile is all about. Also, Agile has to be XP and Scrum together, one without other leads to failures.

Two issues here... first, if you implement just Scrum "by the book", you will likely run into problems at the technical level. This is what Ken Schwaber said, and I blogged about in Scrum is Not Enough. There aren't that many teams that have the maturity in terms of technical practice to implement just Scrum by itself and avoid a mountain of technical debt. As for XP, I would suggest that a team new to Agile implement it by the book, assuming that book is either XP Explained 1st Edition or XP Installed. Do that for 1 release, then look at how to tailor the process to better suit your team or your circumstances. This is the Shu-Ha-Ri method of learning.

Finally, "XP and Scrum together"... um, is that not just XP with Scrum terminology? Scrum instead of Standup Meeting. Sprint instead of Iteration. Backlog instead of Release Plan? As I've said before, Call it what it is, people!!

Dave Rooney
Mayford Technologies

Re: Agile has to be XP and Scrum together, one without other leads to failu by Joakim Holmbäck

In reply to Dave Rooney:

While I concede that XP and Scrum are very similar, albeit with different terminology, Scrum can be applied to much more than projects involving system development. In fact Scrum can successfully be applied to almost any part of a company doing things that does not even involve computers.

When it does come to programming however, in order for Scrum to be successfully implemented you do need to involve continuous integration etc to be able to create things quickly without creating/adding extra technical debt.

Henrik Kniberg describes it very well in this blog post: Introduction to Lean Software Development

Kind regards,
Joakim Holmbäck
xgile.se/

Scapegoat needed? Why not choose Scott Ambler! by Robert Lauer

Why blame Scrum when you could blame Scott Amber. After all, the guy says that TDD is not agile because the folks that fill out his surveys don't do it. Whoa, what a convincing argument!

Maybe we should just stop looking at teams that f**k things up and just try to be better (and James Shore did a nice job at that with his book).

Re: Agile has to be XP and Scrum together, one without other leads to failu by Dave Rooney

While I concede that XP and Scrum are very similar, albeit with different terminology, Scrum can be applied to much more than projects involving system development. In fact Scrum can successfully be applied to almost any part of a company doing things that does not even involve computers.

So? What does that have to do with this?
When it does come to programming however, in order for Scrum to be successfully implemented you do need to involve continuous integration etc to be able to create things quickly without creating/adding extra technical debt.

Right - so call it XP, then, rather than just Scrum.

Dave Rooney
Mayford Technologies

The only thing that matters is the team by Bruce Rennie

Agile methods are not a recipe for success, or anything else for that matter. They're simply a support network that can help a good team, with good intentions, be better.

Note the last phrase: good team, with good intentions. I don't mean a great team, but I do believe you need to start with a good team to succeed with any endeavor. And good intentions. I've seen many a good team mess up big time and usually it was due to the fact they just weren't engaged in the project. The discipline in XP can help a team stay focused, but it can only augment what's already there.

So, for me, those are the fundamentals for success with agile methods: start with a good team and make sure their intentions are good.

And by the way: those same rules apply for any methodology.

Re: Yet another clueless manager by Ilya Sterin

"A bad system will beat a good person every time." -- W. Edwards Deming


I disagree completely. Maybe an automated system is more reliable than a good human being, but when intelligence and artistic abilities are involved, no process can replace it. It might be true for building some departmental applications which are just repeating of what was already done at some other company and though a process could be extracted. But I've mostly worked for innovative startups and no process can ever replace human intelligence. That's why so many big consulting firms are failing, they think they can just put a bunch of code monkeys straight out of college in a room and tell them what to code.

Re: Yet another clueless manager by Matthew Heusser

It might be better to paraphrase deming than to quote him:

"A good person, working in a system that incents him to bad behavior, will become a bad person eventually."

Example: Measure me by lines of code and bugs fixed and see what happens.

My experience with crappy agile is with companies that live in areas that lack a world-class CS school, that hire off the street, don't pay well, and provide a cubical-like existence with heavweight policies and procedures. The programmers they get are, well, rarely go-getters. What's worse, in some bureaucratic organizations, the incentives are more to be political than to excel at programming. It's no great surprise that these organizations struggled before, so they 'Adopt Agile', or "Go Agile", or whatever, but only take the easy pieces, and often insist of still having everything planned and scheduled up front. After a few projects, the business has seen a marginal productivity improvement and the programmers say things like "My job sucks less."

As always, the work you get out of it is a function of what you put in. For the work the people are willing to put into it, that's actually a decent return.

People, not Methodology. by tony siciliano

"..doing agile so poorly that they will almost certainly fail"

Doing agile completely, poorly, or not at all, has not much to do with success or failure.

Actually, most of the successful projects that I know of, did use Agile poorly, borrowing a few ideas from the methodology and throwing out everything else.

Re: Not a condemnation of Scrum by James Nicholls

The biggest problem isn't Scrum or even CSM training, it's people eating dessert and skipping their vegetables.


Exactly. Like all good practices, the problem is people doing it in a robotic mindless fashion. Process for process' sake.

Re: Agile has to be XP and Scrum together, one without other leads to failu by Mark Levison

Dave - a story that might explain why I general don't lead with the language of XP. A few years ago when I was first learning about Agile and doing my reading several of co-workers (all developers) saw what I was reading: eXtreme Programming, Test Driven Development and Pair Programming. Their reaction: that's crazy stuff, you're not going to try introducing that around here. Management was a little more open but also saw it as odd. Finally I read the XP mailing list and felt like I was in a room of fanatics (circa 2002/2003). At that stage I got turned off.

I started reading about Crystal (hello Alistair), but there was no community behind it (the mailing list had 4-5 people) and then I stumbled across Scrum. Perfect no. But it gave us a language to use that didn't scare management or the developers. Now several years and two acquistions later I'm helping to introduce TDD to a development organisation in excess of a thousand people.

If I had tried to sell XP I probably wouldn't be in this position today. So like it or not language matters and you have to go with what resonates with people. Today if I were talking to Sr. Management at an organisation I would talk about quality and waste removal. To the mid level mamagers I would talk about Agile (generic) and to team members both Agile and Engineering Practices.

Finally it helps to remember that the engineering practices predate Scrum and XP - if my memory serves they come from Kent Beck and the smalltalk world. So maybe we should say: Scrum and Smalltalk Engineering practices.

Cheers
Mark Levison
Notes from a Tool User

Re: Yet another clueless manager by Clay Nelson

Whether you call it process or just shared values in action, smart people have to have agreement on how to collaborate with each other. Plenty of teams with smart people have failed in more spectacular ways with few morons around.

Software development is both art and science. The key is giving lots of creativity up front when exploring solutions and applying more engineering and economic rigor later in the development cycle. Also, understanding how much novelty you are dealing with determines how much to balance between the art and science. The more business model and/or architectural unknowns the more need for flexibility at the outset.

Re: Yet another clueless manager by Clay Nelson

"the illusion of understanding" is when people think they know what you mean but really don't. The same thing occurred with RUP in the late 1990's. RUP was always meant to be iterative with a right sized amount of artifacts depending on the size and distribution of the team. Unfortunately, way too many people said they were "using RUP" but really had just renamed some of their documents and kept on managing projects the way that had.

We as a community must be realistic about the cultural and operational limitations that exist in many larger shops that make quick adoption of the risk mitigating properties of agile development difficult. Agile can be applied to many situations, but not all (at least not immediately).

Re: Gartner hype cycle by Gregory Mostizky

I think this is directly related. Please see my post on the issue: Agile Hype Cycle

The Decline and Fall of Agilists by Jurgen Appelo

I wrote my own reply to all agilists who are preaching that certain agile practices are required to be called 'agile':

The Decline and Fall of Agilists
www.noop.nl/2009/02/the-decline-and-fall-of-agi...

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

24 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2013 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT