Uncovering Serious Flaws of Agile and Scrum

This is an intriguing position for Kai to take..... especially as Evo is considered to be an Agile methodology.

Many Agile peeps agree that the manifesto is out of date. It does not fully reflect the current views of Agile. I discussed changing the manifesto with Alistair Cockburn in 2003 to say "Business Value OVER Working Software OVER Extensive Documentation". I now agree with Alistair that the manifesto is a historical document, a call to arms and should be left as such. It took a few weeks to draft the original with just a handfull of people. Can you imagine how long it would take to draft a new version with thousands of different opinions.... And to what value?

You only have to scratch the surface of Agile to discover that it is obsessed with business value. For the past to two years, there has been a "Customer and Business Value" stage at the main Agile20xx conference in the USA. Of the presenters, David Rico has written a book on Business Value, and Dennis Stevens has been published in Harvard Business Review. Ryan Shriver has presented a number of time, often incorporating Tom and Kai Gilb's work.

The "As A xxxx I want xxxx" format was upgraded to "As a xxxx I want xxxx So that ( business value )" in about 2003/4. For the past couple of years Liz Keogh has been promoting the "In Order to ( Business Value ) As a xxxx I want xxxx" which place business value as the primary concern.

I myself have been promoting an approach called "Feature Injection" which was originally called "business value driven development" or somesuch.

Agile is obsessed with Business Value. Rather than base an opinion on a document that is years out of date, I would suggest people go to an Agile Conference or join a yahoo group to find out what people are really doing about business value.

I've only read the intro and the Part 1 "Wrong Focus" so far. The most important line from the article I think is:

"It is not about working software. It is not about your preferred working process. It is not about user stories. It is not only about your customer or user."

It does strike me as frustrating that it feels like we always have to "explain" the agile manifesto. It should be more clear about delivering ROI to customers / users.

However, to be fair, I think most of the value that has come from Scrum in particular, and Agile in general, has been to help developers and customers learn to work in an incremental fashion. This is crucial. It is definitely important to start thinking about delivering incremental customer value specifically, not just "working software", but again to be fair, "working software" has an implication that it's "working to the definition of the customer" in my mind.

I wrote a blog post recently about how agile software construction is more like building a complete neighborhood, rather than a single house. The idea being that if a small team of building just building "working foundations" one after another they would never complete a single unit of value for the company or any buyer / user. Only when they complete an entire, complete, working house is that unit ready for inspection / sale. From there, they can continue to build more houses, but will have gained both critical experience / feedback, and ALSO put the business in a position to make a sale and gain ROI.

Here's the post: agilefromthegroundup.blogspot.com/2009/12/agile...

Looking forward to reading the rest of the series.

Reading this article I can only conclude one thing: Kai didn’t really get the scrum philosophy (and misinterpreted the agile manifesto).

How can you look at the standard scrum process diagram and have trouble identifying the business value? At the end of every sprint new features and improvements are delivered. So every couple of week (depending on your chosen sprint length) the system improves and can be used to generate business immediately.

The agile manifesto states that working software is more important than documentation. Working software does not mean software that compiles or (almost) never crashes. No, working software means software that does what it is supposed to do. Software that fulfills the requirements of the business. So yes, working software in this context means adding business value.

As for the programmers, instead of losing freedom, they gain a lot of freedom with scrum. In the typical waterfall project you'd see a huge requirement/design document stating almost literally what the developer needs to program (sometimes even already written in pseudo-code). The user-stories in scrum on the other hand are very small and limited. They require the developer to discuss them with the product owner (business) and fill in the smaller details during the sprint. If you don't like the way a user story is specified, scrum allows room to discuss the user story with the product owner and use your creativity to come to an elegant solution together. If the business state they want a "shopping cart" you can ask why they think this is the best solution. With waterfall projects all requirements are set up front. So if the business put in the requirement that they want customers to be able to buy books the programmer puts his imagination and freedom to work and creates something he thinks is very elegant. After a year the software is delivered and the business is in for a surprise. They expected a simple "shopping cart" solution but got a complex, fancy system which works completely different from what they expected. So yes, the developer felt like he had a lot of freedom but in the end the added business value is almost none and the feeling of freedom was only created by incomplete specifications.

So yes, in a nutshell scrum is a to-do list. But instead of losing freedom programmers gain lots of it. And in the end the business gets what they really want so business value has the main priority.

Of course scrum is not without its pitfalls (scope creep, lack of business involvement) but when applied correctly will lead to the best working software resulting in the most added business value.

Kai says he's talking out of extensive experience, so assuming he's not making this up, there seem to be some problems with the typical agile project.

But how does that make Agile itself flawed? The XP projects I've seen were delivering nothing but business value. Iterative development (the chance to change your mind as you learn) and planning with realistic estimations (is it worth the cost?) are all about business value! And then there's a lot in XP that makes sure that business value is not only aimed at, but also delivered.

So if you don't make your customer happy in terms of ROI, changes are it's not because your doing it the Agile way.

In my experience, XP does create an environment of creativity that lets people work toghether towards business value. Yes, it's also about micro mananagement, but not by managers, but by the team itself. Baby talk requirements? Stories are invitations for conversations, there's enough room for creativity between customers and developers.

But XP is focused on customer projects, so if you use it for product development you have to adjust it a bit. It does not cover product management, obviously. Guess what, it does not cover marketing either. Is that a flaw too?

(Can't talk too much about Scrum. In theory, it covers only a very narrow part of software projects, somewhere between requirements engineering and development practices. Real-world adoption may be different. But Kai speaks about XP too.)

All in all, I think Kai might have a lot to say about business value and requirements, but packaging that as flaws of Agile seems ill-advised to me. Even if it gets him more attention than just pointing out what might be required as well.

I'm wondering if Kai is actually trying to "Punk" the community by getting people to talk about it. It's the sort of reverse psychology that just might get some honest and frank discussions about the resistance met in many organizations still entrenched in waterfall.

Others have made excellent points, and VALID points about where value to stakeholders is introduced and compounded upon. If Kai's not doing the reverse psycho shuffle, and he's genuinely not seeing where value to stakeholders is coming from, then perhaps his definition of "Value to stakeholders" and everyone ( or at least everyone engaging in him responses ) elses are at odds.

My mother used to say "there is always a convincing reason for an action and a real reason for one". Kinda sneaky, but there it is.

I have an emotional response to this piece based on attending one or more Evo presentations. I got the distinct impression each time that there was a pitch for this process as the "right" solution.

So, back to my mom's words of wisdom. The convincing argument is that Agile focuses on the wrong thing (although not that convincing when you read the well thought out posts below).

I'm suspicious as to the real reason - yet another way to pitch Evo.....

Well, actually 2 points. Firstly, generally when people have issues with Agile - it's not as much an issue with the "spirit" or intent of Agile as it is the particular implementation. Often the complaints I hear are absolutely legitimate. But they are also often a result of certain common anti-patterns, either in adoption or implementation of process (e.g. too much dogma, not enough thought).

In the example of the story above: "As a buyer, I want to place a book in the shopping cart".

I agree that this could potentially be viewed as overly constrained. As Martin Fowler points out in this post (martinfowler.com/bliki/ConversationalStories.html),however, stories are supposed to be conversations, not decrees. As a developer we should be able to, and indeed have an obligation to engage in this conversation. I.e. we should be allowed to ask "why do we have to have a shopping cart?". If we as developers feel that we cannot have this type of dialogue openly, then my feeling is that there is a deeper dysfunction - for example, lack of trust and/or mutual respect - that *no* process, Agile or not, can overcome.

@Ho Yan: "If we as developers feel that we cannot have this type of dialogue openly..."

This is precisely the thing that is at the heart of the subject. What Kai, and I suppose by inseparable association, his father Tom, have proposed is about control and

If one were to look at the Evo summary ( bit.ly/dfFyxD , for those who care to look ), and walk through the 10 Evo principles, you can almost drop the word "Scrum" for "Evo" in the gross majority of the points, and you'd almost swear you were looking at a summary of Scrum instead.

The one thing you cannot say about Evo is that it's open. It itself is a closed methodology. I know of no community process, or other organization that
drives Evo's adoption by open inquiry and adaptation, so Amr's Mom might very well be onto something there ( Thank your Mom for us, Amr ) !

Taking a look at just three statements from the Evo Principles doc:

Item 8 : "Evo is a discipline to make us confront our problems early"

Funny, that sounds like Scrum to me.

Item 7: The Evo project team will focus their energy, as a team, towards success in the current Evo step.

Substitute "step" for Sprint, and we're back to Scrum. Interesting pattern developing.

Item 10: Evo should allow us to prove out new work processes, and get rid of bad ones early.

Deja Vu all over again here. Continuous adaption to using what works, ditching what doesn't. Sounds like Agile/Scrum to me.

Wow, so Scrum ( and by association, Agile ) has flaws, but no really serious ones so as to not to be able to literally LIFT concepts en-masse, massage them, and repackage.

I guess that old quote about the sincerest form of flattery is true after all.

Apparently I DID get into trouble for saying that. I posted a reply last night that seems to have gone AWOL.

At the risk of it happening again, my reading of the blog posts felt like I was locked in a room with a telemarketer or an infomercial guy (Slap! Chop!). This is one aspect of the agile community recently that I really regret. If you can't present your point without a lot of emotional manipulation then you don't have much of a case.

Now, that said, I don't think Mr Gilbs points are completely without merit. He seems to have been around agile long enough to know that some of his statements are not totally correct, so I interpret his text to be more directed at those of us that have become more caught up with making the burn down chart look pretty than with delivering value. Fair enough, but this behavior is not restricted to agile methods.

Anyway, I pretty much agree with your view: a sales pitch.

I experience on few latest projects this problem, that testers or end-users already "know" what they need and create "directive" issues, that make mislead to developers or can be totally wrong ideas. I also had feeling like we are chasing to much to-do list and creating pandora box by just adding and adding new features.

But is this a problem of Agile concept or of a poor lead of project manager (scrum master) who doesn't know to focus communication to important things and lead retrospectives (and other meetings) trough ruff zone to a best solution.

If you want to add most value to stakeholder, set a highly unrealistic deadline. Let's say developers must finish one year project in 1 month only. That give best value to stakeholder, but at what cost?

The best thing is balance, and agile I think already met the balance well. Nothing wrong with agile, only stakeholder think that developers should work 100 hours / week to increase stakeholder's wealth

Amr - I'll join in getting trouble here. It feels like a slamdown - i.e. the Scrum/Agile community get XXX and YYY wrong - oh by the way we have the solution its called EVO.

I think that the ill defined role of the PO is a weakness in Scrum but I think that Kai's comments blow it out of proportion here.

Cheers
Mark Levison
Agile Pain Relief Consulting"

The Gilbs have always placed an emphasis on the importance of the way requirements are specified. They have an approach which also makes much of the "non-functionals" for a requirement: I shouldn't specify I want to place a product in a shopping-cart, I should specify that I want to increase sales of books by a certain percentage/hit a certain target, for instance.

Looked at in that way, there is a greater freedom for the 'developer' (or team, since this would certainly include UX, etc.) by stating the problem in terms of measurable outcome of business value.

Having been in a workshop with Tom and Kai Gilb at the company I work for a couple of years ago, I found that way of specifying requirements interesting, but very hard for the business owner to work with. The business *should* ideally have a solid business case for a requirement, but often this is simply: "I want to have the same as all the other companies have, so customers will find it familiar", and it will not be seen as worth the effort to do it the 'proper' way.

That said, too often this simple view is taken even when there would be much worth in specifying a measurable level of success. Even large project are run without any idea on what the ROI would be, and how to measure that. Introducing a new web-shop, for instance, can have requirements meant to increase sales, improve the company's image, and lower costs in product catalogue management. Simply stating those goals clearly can make a big difference in the focus of a project, but keeps the possible solutions completely open.

Anyways, it seems Ok to me to say that you could probably create better solutions by stating your goals more in the terms of the business value you want to achieve. It just doesn't work out that way in many cases.
And is doesn't mean Agile is flawed (not that it is flawless), but it does make a nice challenge for a product owner.

> As a buyer, I want to place a book in the shopping cart. [End of story!]

No Agilist I know worth his (or her) salt would ever write such a lame story. As Chris Matts points out, there is no value statement. Not only that, it describes a "how" which is not how user stories should be formed. To take a really bad example of a (so-called) user story and use that as an example of why the concept is bad seems like cheating. Heck, it /is/ cheating. Come on Kai, you can do better than this if you want to criticize! Show us what you are made of ;)

> The pure Function is “to order a book”.
Actually, you don't know that. Sometimes in the supermarket I put things in my basket to compare them with other things later. And sometimes I just change my mind and leave them at the checkout, or put them back. Breaking a "pure function" down into smaller pieces allows creative solutions to emerge. If we only focused on "I must buy this book" we potentially lose many other useful "change-my-mind" functions along the way. I agree that we don't want to write stories that tell developers how to solve the problem (e.g. put in shopping cart) but we also don't want to restrict the solutions to the obvious one that we /think/ should occur. Small stories are good for innovation.

Seems he can not help pitching about his "great pieces..." Interested? Go check out @kaigilb...

s

Thank you! We are so obsessed with delivering business value we had to create a manifesto which, in my opinion, attempts to remove the obstacles in our way to doing so.

I welcome Kai's provocative observations, they guard against groupthink.

It seems odd to refer to scrum as "orthodoxy", since some would still consider it bleeding edge. But certainly in some circles scrum has become a bit religious, if you'll forgive the term.

Scrum is very team focused, and sometimes we forget that success is about the customer, not the team. To me, the revelation of well-executed scrum is its ability to draw out the best behaviors of the dev team, thus the best results.

In the end, delivery of superior business value is not a result of any process, but of organizational focus, insight, and leadership, which come from people. Given that these factors are in place, I believe that agile/scrum is a superior way to develop good software as quickly as possible. Without those factors, it doesn't matter much how you develop.

Of course, it is rather an example, which is dangerously moving to a "reductio ad absurdum"-state in the article AND in the comments. It was taken from a primer as an example, where it was even between parentheses, not teaching the way proper stories have to be written, but rather a sidenote in a primer to SCRUM.
The fact is, that a lot of user stories are written poorly, but this is more the problem of writing good user stories, instead of doing SCRUM/Agile well. SCRUM is very open, and every practice can be taken into SCRUM (hell, if Use Cases would serve its purpose there, by all means, use it). Fail fast will prove that practices should be abandoned quickly. Of course, quality attributes tend to not fail fast (you have to carefully construct the sprint backlog to fail fast e.g. portability attributes)

'agile' has been perverted by the same management obsession with dates and unwillingness to pay for quality that destroys all software development

managers cherry pick; they hear about 2 or 3 week iterations and not much more, because 2 or 3 weeks sounds a lot better than many months or years

but programmers are also at fault; too many are content to just hack, and 'agile' gives the excuse to not think, but just code, and put off dealing with anything complicated ('the backlog')

of all the so-called agile practices, scrum is the worst; daily standup meetings are a gimmick

the truth that many developers and almost all managers dont want to face is that there is nothing free

software is complcated and thinking is a necessary part of it, and understanding requirements is absolutely necessary

being cute about 'stories' and 'sprints' etc does not replace the hard work of communicating complicated ideas among many stakeholders and doing disciplined analysis and design and implementation and testing

the details of how these things are done can vary, but they must be done

phloid, I invite you to join our team of agile thinkers and developers, who put professionalism and craftmanship before anything else. You will be amazed by the success stories (pun intended) they can put forward. They never claim you should not think about quality, communication, discipline, openness etc.. In fact, these are indeed powerful values of any agile team. Hmmm... I seem to forget another important one: RESPECT.

James

wow, so much negativity

And this Kai guy is trolling...where is the value delivered? LOL