The Flexibility of Agile: Flaw or Strength?
The principle of “responding to change over following a plan” is considered by many as a strength of agile. But some consider it to be a flexibility that can’t work in practice. They have seen agile projects that had difficulties managing changes and customers who expected too much flexibility, leading to project delays, quality problems and teams that had difficulty delivering value. Can agile not live up to its promises, or is it the way that teams and organizations have adopted agile that is causing the problems?
In the article why agile isn't working on CIO.com, Lajos Moczar states that he thinks the agile methodology is flawed:
Agile promises many things, but the reality in the field is often very far from the expectations.
I've concluded that agile has not only failed like other fad methodologies before it but, in fact, is making things worse in IT.
One of the flaws he discusses is “development over planning”:
In theory, developers code while collaborating with stakeholders to define, refine and change requirements as the projects goes along. The methodology, however, does not distinguish between big and small changes. Every change has a cost, but agile does not account for this. The result?People often change really big things late in the game using the rationale that since it's an agile project, it can handle it. The only way the project can handle this is by adding iterations. As that happens, defects that might have been easy to fix at one point get harder and harder to fix, since the code base keeps changing.
This is much different from more traditional practices, where you have a project based on well-defined requirements and changes are managed through a Change Management process—which, while sometimes byzantine, will at least state time and monetary costs more clearly.
Shane Hastie reacts on the article from Lajos in his blog post please don’t equate tragile with agile:
The author misinterprets the agile manifesto and takes some of the agile practices completely out of context, using unfounded statements to justify his conclusions. This post is my response, addressing the points he makes and tackling the myths he propagates.
He reacts on the flaw “development over planning” that Lajos mentioned by explaining the problem he sees with up front defined requirements and change management:
The reality of the business environment today is that business needs are volatile. The “half-life” of a business requirement today is approximately 6 months, so if you’ve identified what is needed and it takes 12 months to deliver it then 75% of what was asked for will be wrong, purely because of the passage of time.
Agile addresses the needs of customers in a disciplined and careful way, and doesn’t oppose to planning as Shane explains:
Agile is not an excuse to ignore good design principles, nor to allow massive change without doing impact analysis. Agile is about adapting within boundaries, and recognising that when change is so large that it impacts the business value of the project then it needs to be assessed and evaluated, but not resisted.
There is nothing in the agile manifesto about not planning the overall structure of the product from the beginning, and certainly none of the respected authors and thought leaders in the space recommend starting to write code “like free form poetry” – any project needs SOME design up front, and in agile projects we resist BIG design up front because we know that the world will change around us.
The article why your users hate agile development (and what you can do about it) by Esther Schindler on ITworld talks about the different views that developers and users can have of agile. One of the difference she discusses is “you say iterative, they say disorganized and never-ending”:
Early in Agile's history (…) many shops saw Agile as an excuse for development teams to avoid providing estimates, documentation, and plans. Things are far better, nowadays, but I'd argue that these bad reputations lurk far longer than we prefer -- even in the development community itself.
Agile teams often try to be as flexible as possible towards the needs of the users, but users might expect too much from this flexibility, making it difficult for teams to keep on delivering value:
Sometimes that "Agile means disorganization" mindset encourages users and clients to interpret the process as "It's okay to be capricious." Yes, Agile does permit users to swap priorities even towards the end of a big project. But developer Sorin Costea observed the downside when things worked smoothly: "These users could not believe there are also limits to the delivery power. They could not really grasp velocity or timeboxing, but expected to throw in stories any moment and also get them in time." With the reasoning that, "Hey, we are Agile aren't we?"
Corbin Norman blogged about acknowledging the common good of agile: praising the good of agile and abrogating the minuscule negativity it brings. He recognizes how people sometimes have a negative view of agile:
Critics of Agile say this development methodology is making things worst for some software people by promising solutions it cannot deliver and displays a habit of promoting sloppy requirements, hiding true cost of development and preventing effective management. Their argument is the latter leads to long-running projects, dissatisfied customers and an overall IT ineffectiveness.
Today, there are quite a few arguments to be made on both sides for the practice of agile development but the community should take notice to the amount of good Agile is doing rather than the minuscule negativity it brings.
According to Corbin, one of the good things from agile is it’s flexibility:
With Agile, schedule and cost are the major determining factors and it is scope that changes in order to accommodate acceptable business demands…the client’s needs. For mature Agile teams, this level of flexibility is brought about by years of experience working together. Traditional Waterfall won’t allow this level of flexibility once a project is underway since it strictly adheres to the concept that once a stage/process has ended, one cannot go back to adjust.
Agile doesn't mean necessary faster
"Individuals and interactions over processes and tools"
I've found that it pays to learn "tricks and tools of the trade" (as in xp practices, kanban principles, etc.), it pays to find and cultivate the right individuals and interactions, but processes are a hard fit. Yet it's one of the first things projects chose - by saying they'll do Scrum, XP, or whatever. Processes ought to be custom fitted to the people/project/organization at hand, but they all contain some variant of the "Thou shalt have no other processes before me" commandment. Second to the "Let's use the same process in our entire organization, so we can easily transfer people around and benchmark them".
Further, processes ought to first spell out under which conditions they are a good fit, and under which you will incur extra costs (yeah, the part that's always underestimated by its proponents). They all claim near universal applicability, except some absurd situation you can't possible agree with - such as "... if you aren't going to have changes ...", "... don't mind the quality...", or "... have bags of money and oodles of time to spend...".
So, is Agile oversold? In my opinion, yes. When you have developers and product owners quoting the agile manifesto to each other in order to determine priorities, and bring weight to their argument, then you're probably heading towards the dark side of agile, where ideology trumps pragmatism.
A good step forward would be to abandon the "No true scotsman" arguments prevalent in Agile discussions about failed agile projects, and instead present the processes as templates: "Here you go, this set of tools might work for you. Do WTF you like with it". May be you _are_ special (why else would you be doing it as a project?). People should be proud to do Scrumbut, and not shy away from telling others how they changed the rules and made stuff work for them.
Recipe for Disaster
For starters, they present (as is often the case with agile) a false dichotomy:
1) Either you value change more than planning or....
2) You use "waterfall" and cannot change a thing
Clearly, #2 is false, since there is no real waterfall process, just traditional ones, and those have always had some mechanism for change.
In terms of responding to change as a default mechanism, not only is it reactive, and thus backwards looking, it invites large changes late in the game.
Not only that, but coupled with "potentially shippable product" -- eg, no prototyping, means the changes are affecting huge code bases. No wonder people are going crazy.
Feel free to check out postagilist.wordpress.com/2012/05/24/responding...
Risk management, technical product design, and Agile
In my short experience within Agile, I see neither of these as problems of Agile, but rather as problems of management, and actually Agile theory aims to avoid both of them.
Firstly with regard to deferring big changes, this is absolutely counter to Agile. The highest risks need to be addressed first according to Agile. Although the best I can say of our own Agile practice is that it is patchy, we have stuck to this rigorously, with the result that areas which require significant planning and change get managed early, before they become crises. It is hard for me to even imagine a development calling itself "agile", while deferring high risk decisions.
As for lack of analysis and design in an agile development, to me this simply smacks of bad discipline by the product owners. Again, I consider us to be a fairly patchy agile implementation, but analysis and design are essential to the development process. The biggest difference in agile is that we don't do the design until we need it, so that means it is more likely to be current when it is done.
If there is one thing I always hated as an analyst/designer, it was building a really excellent comprehensive design document which was abandoned long before it ever got to the development team, as reality changed by the time anyone was in a position to implement it. Never happens now we are somewhat agile, but a lot of design is written down and published nevertheless.
One area where we diverge on purpose from standard agile practice, is we have technical skills in the product owner team. My title is product architect/b.a./product owner, and I work hand-in-glove with the product manager/product owner on the one development. Mine is a full-time role while his product manager role takes up more than half his time, though his time in non-technical product owner tasks is essential to the development process.