This article is part of the Agile Manifesto 10th Anniversary series that is being published on InfoQ.
For the first seven years of my career, I worked on projects which still retained much of their Waterfall habits. For seven years I sat at computer screens and tapped out code - in basements, late into the evening, sometimes at weekends - and not one thing I wrote in those seven years made it to production. Projects had endless reams of features added, were restarted, were shelved, and in one case made it as far as litigation - but no one used a thing that I wrote in those seven years.
So when I came across Agile for the first time, I was, like a lot of newcomers to the movement, enthusiastic. "You're doing it wrong! That won't work!" I promised, looking at companies who had failed to embrace the new, incremental, collaborative way of creating software. It was the prevalent attitude of the community at the time. A prophecy had been written: that companies who failed to change would fail to survive.
Ten years have passed since the manifesto was written, and still many companies are failing to deliver effectively. Some of those companies are "doing Agile". Others are still surviving with their year-long projects and up-front analytical methods intact. But they're surviving.
The Agile manifesto starts, "We are uncovering better ways of delivering software by doing it and helping others do it." For the contexts in which the authors were applying Agile methods, they worked. Mostly these were small, co-located teams with the right mix of skills to succeed. Over the last few years, increasingly we've applied Agile to larger projects and organisations. These Enterprise behemoths change more slowly, if at all. Richard Durnall wrote a post on Agile adoption patterns, showing how each aspect of the business breaks and is forced to change to support the Agile adoption - the people, tools, governance, customer, financial controls and organisational structure. More often, though, we're seeing the structure of companies hold firm against the change process. People retain Waterfall habits and thinking, creating deadlines that are often arbitrary and holding development teams to them. Tools are used instead of conversation, rather than to record and support it. Governance bodies still ask for certainty that's impossible to achieve over the lifetime of a project. Customers sit in one office with developers in another, unable to properly communicate or give feedback. Financial controllers insist that analysis is done up-front, theoretically ensuring a good return on the investment of the budget.
Part of the problem we face is that traditionally, Waterfall has involved "bicycle thinking". It starts from the premise that a software project can be decomposed into small pieces of analysis, which can then be implemented, and those implementations reassembled into a system.
Unfortunately, a "software project" isn't really made up of the analysis or its resulting implementation. It's made up of the people involved in the project, their ideas and interpretations, in a context which has been created by other people around them. Its properties are not those of the people, though. As Jurgen Apello points out in his book, "Management 3.0", the properties of a system emerge as a result of the interactions of the elements within the system. Agile can't succeed as long as the environment stifles interaction or rewards simpler interactions like those up and down a hierarchy.
If complexity is going to be happen anyway, we have to allow those patterns to emerge from the interaction of people on projects, and from the interaction of those projects themselves. We're guilty, as a community, of signing up to "individuals and interactions over processes and tools", then mandating processes to control the interactions, while supporting the processes - and not the interactions - with tools. In future, the practices we teach will be those which enable interactions, rather than controlling them. We've seen this already with the rise of metaprocesses like Kanban. Models for understanding complexity, and particularly the complexity of people, are also being taught in Agile conferences worldwide - Systems Thinking, Complexity Thinking, psychology and sociology.
These are also the kind of practices we need to change behaviour at higher levels in the organisation; to make the impact of anti-patterns apparent. We haven't seen as much change as we prophesied at the beginning. Maybe over the next ten years, we'll see a different manifesto emerge - one which starts, "We are uncovering better ways of enabling change by doing it and helping others do it."
About the Author
Liz Keogh is an experienced Lean and Agile coach, trainer, blogger and well-known international speaker. Coming from a strong technical background, her work covers a wide variety of topics, from software development and architecture to psychology and systems thinking. She is best known for her involvement in the BDD community, and was awarded the Gordon Pask award in 2010 for deepening existing ideas in the space and "coming up with some pretty crazy ones of her own".