We are now at a crossroads in the Agile-adoption narrative. Early in the story, teams were the “bottom-up” vector for Agile spread. In the middle of the story, the way Agile spread started to shift away from teams. Executives and “management” began serving as the new vector for the spread of Agile. And a bit later in the story, not too long ago, the next shift got underway.
Since 1995 or so, we have progressed through most of the stages of the classic adoption curve: Innovators (2.5%) , Early Adopters (about 13.5%) , Early Majority (about 34%) , Late Majority (another 34%) , and finally…the laggards (about 15%). We appear to be well into the late-majority phase. The next stop is “the laggards.”
How did we get here?
What have we learned?
What are we doing about it?
One more recent development is the creation of larger Agile consulting and "Agile coaching" groups. These are consultancies with a focus on bringing "agile" practices to the "late majority." Most of these "late majority" adopters of agile are larger enterprises that struggle with change.
What does this mean?
Where are we?
Where might we be going?
Before the Manifesto: from about 1993 to about 2001
Taking stock of history may provide a guide.
From 1993 to about 2000, many software engineers were doing many experiments around the world. All kinds of people were doing all kinds of experiments to try to bring more predictability, reliability and quality to the process (or “game”) of software development.
(NOTE: Those who enjoy history can find a wonderful chronology of Agile history at the Agile Alliance web site. This document traces the early history of Agile in some detail. You may also find a wonderful family tree of Agile practices here.)
The Scrum framework was also developed during this period. Jeff Sutherland and Ken Schwaber, working in the Boston area, conducted many experiments focused on improving the results of software development efforts. You can examine this history by examining the Scrum Papers on Jeff Sutherland’s SCRUMINC web pages.
Jeff and Ken (like many Agile Manifesto signatories) conducted many social experiments inside many software teams. These included some experiments with all-new ways of working. The experiments included expecting teams to try to: execute a short daily meeting; and work towards frequent, predictable deliveries; and to execute periodic “retrospective” meetings.
During the early development of Scrum, teams were expected to give it a try. The result was Scrum: a set of roles, events, artifacts and rules that we collectively call “the Scrum framework.” This collection of practices called “Scrum” tended to strongly support the core principles of what later came to be called “The Agile Manifesto.”
From about 2001 to about 2006
During this period we saw the creation of the Agile Manifesto, the end of the “innovator” phase, and the birth of a new role: the “Agile coach.” Concurrently, we also saw the beginning of “The Agile Imposition….”
2001: The Agile Manifesto
The Agile Manifesto was drafted by a set of people who got together at a ski resort called Snowbird. These people who were doing experiments in new ways of testing, new ways of programming and new ways of organizing teams. These individuals got together and agreed on some fundamental things. And they created something important from those agreements.
What they created was a simple document. That simple document specified 4 core values and 12 related principles that formed the basis for what later became known as ‘agile software development.” This document continues to serve as a core part of the definition of “what Agile software development is.”
2001 to 2002: The End of the “Innovator” Phase; and the Beginning of the “Early Adopter” Period
The year 2001 brought the creation of the Agile Manifesto and marked the end of the ”Innovator” phase It also launched us into the “Early Adopter” phase, where a whole new set of participants came in.
After 2001, some conferences started focusing on ‘agile’ software development ideas. Up to this point the original Agile Manifesto group called themselves “the Agile Alliance” loosely, for about a year after the signing of the Manifesto. Then in late 2001 or early 2002 Ken Schwaber asked if he could make a not-for-profit corporation called the Agile Alliance, and the group gave him that permission. The previous, informal "Agile Alliance" of the signatories dissolved, and the new, corporate Agile Alliance run as a corporation came into being.
It was at this point that trend-aware software developers worldwide started paying attention. These developers examined web pages, attended conferences and purchased books. They socialized these books to colleagues, and eventually, a bottom-up movement began taking shape. Teams learned about agile practices and gave them a try, in pursuit of professionalism and continuous improvement. What is important to note here is that these teams consented explicitly to try these new practices. There was no “mandate from authority” or coercion from “higher ups.”. These early-adopter teams throughout the world were opting-in (“consenting”) to the new game of “agile software development.” This was a more or less “bottom up” phenomenon.
Over time, the trend in agile adoption shifted, from “bottom up” to “top down.” This trend shift was subtle at first, and occurred from about 2004 to 2007 (or so.) The shift came from reports of successful agile adoptions. These reports began appearing in IT industry journals and management magazines and web sites. As Agile software development got traction, IT leaders and company executives began paying attention….
The Birth of Agile Coaching: 2006 to 2008
The idea of an expert “coach” or consultant in Agile practices certainly predates the year 2006. But it was the period from about 2006 to about 2008 that firmly established Agile coaching as a professional role in IT and software development. Typical IT shops in typical organizations began employing “Agile coaches” to help them figure out how to get off on the right foot.
Large organizations started paying handsome “per diem” rates to coaches and consultants who could help their IT teams “be Agile.”
This was different than the previous “Early Adopter” days, when Agile came into organizations from the “bottom up.” At the start of the “Early Adopter” phase, teams just tried various practices, keeping practices that worked well and discarding practices that did not serve well. They all consented. They did not ask anyone for permission. They just did it.
This new stage clearly marked the end of the “Early Adopter” phase and the very definite start of the “Early Majority” phase. Organizations seeking improvements in software quality, predictability and cost began “rolling out” Agile across entire corporate IT departments. These rollouts sometimes employed “embedded” Agile coaches for years, inside single enterprises with dozens or even hundreds of teams.
Agile coaching was becoming a big business.
The Rise of “The Agile Imposition”
In 2006, Agile Manifesto signatory Martin Fowler drafted his essay “The Agile Imposition”, arguing strongly and convincingly that imposing Agile software development practices on teams without their consent was fundamentally at odds with the Agile Manifesto itself.
The following are some quotations from this important essay:
…As a methodology or design approach becomes fashionable, then we see a lot people using it, or teaching it, who are focusing on the fashion rather than the real details. This can lead to reports of things done in agile's name which are a polar opposite to the principles of movement's founders…
…Imposing an agile process from the outside strips the team of the self-determination which is at the heart of agile thinking.
… imposing agile methods introduces a conflict with the values and principles that underlie agile methods.
…I’d rather have a team work in a non-agile manner they chose themselves than have my favorite agile practices imposed upon them.
…So I hope I’ve made clear that imposing agile methods is a very red flag. - Martin Fowler, Agile Manifesto signatory. - Written 2006, the Agile Imposition blog post, Martin Fowler
InfoQ wrote about this essay in 2006 in Fowler: "Agile Imposition is a Very Red Flag".
Since about 2006 or so, many well-meaning company leaders have sought coaches and consultants to help them with a transition away from traditional software development towards the adoption of agile practices.
Likewise, many well-meaning coaches and consultants began seeking the business of company leaders looking to migrate to the new way of working.
During this period, Martin Fowler’s essay “The Agile Imposition” was quietly ignored, and slipped into obscurity. The imposition of specific Agile practices on teams did produce results. Teams using Agile methods saw improvement in all of the metrics they were measuring. Improvement of 20, 25, even 30 percent became commonplace.
From about 2008 to 2014, Agile coaching experienced tremendous growth. There were, however, some serious stress fractures as a result. Because of the highly authoritative nature of imposing Agile practices on teams, stories emerged about Agile adoptions gone bad, after the consultants or Agile coaches vacated. At issue was the “People and Interactions” value of the Agile Manifesto. Did people and interactions actually matter where Agile adoptions were concerned? What imposing Agile practices on teams actually a sound idea?
Was Martin Fowler right?
2008 to 2014: The Transition from “Early Majority” phase to “Late Majority” phase
From about 2008 on, Agile and Agile coaching experienced tremendous growth. Before long, “agile software development” became a very mainstream trend and more and more large organizations began taking agile seriously. Starting about 2012, it became evident the “Late Majority” phase was coming into existence.
If you ask 100 Agile coaches how many organizations actually get an Agile adoption that sticks, the answers may vary. The conventional wisdom is that only about 15% of typical Agile adoptions actually deliver truly transformative results. Agile at scale has remained elusive, even as hundreds of millions in coaching revenues are being collected. Regardless of the actual percentage, we can be fairly sure the number is under 40%. If these numbers are in the neighborhood, then in general, an Agile adoption is not a good bet.
During the period from about 2008 to about 2014, Agile coaching became very big business, with huge expenditures for training and coaching of teams.
Which brings us to the tremendous transactional success of agile coaching, which has been a smashing success in monetary terms for large agile coaching companies.
The Larger Questions
As stated previously, well-meaning company leaders employ well-meaning Agile coaches to (for the most part) introduce Agile practices to existing teams who have not yet worked that way. This typically includes some training, and then substantial coaching.
Now that we are in the Late-Majority stage, what does this mean in terms of the experience, for a typical team member?
Here are some implications for most Agile adoptions:
- Teams and team members are not consulted to discuss what they think and feel about the new Agile practices.
- Teams and team members are expected to use specific Agile practices. These practices are typically imposed on the teams without their consent.
- Teams and team members experience a sense of disempowerment, due to the fact that they play no part in the process change
In Closing: Some Ideas to Consider
The advent of Agile coaching as a profession is transforming the professional services industry.
The recruiting, staff augmentation and professional services marketplaces are still adjusting the to the tremendous growth in demand for Agile professional services. The creation of these larger Agile consulting and "coaching" groups is signaling the beginning of a all-new chapter in the wider, worldwide Agile narrative.
As this new chapter unfolds, organizations that have attempted “Agile-at-scale” previously will begin asking important new questions:
- What have we received for our money?
- Was imposing Agile practices on teams really effective for us?
- If not, why not?
- If so, do we any have “WOW-level” success stories?
- How might we go about this business of adopting Agile in a more effective way?
- Who is offering alternatives to what we have already tried?
- Is human engagement essential for a rapid and lasting Agile adoption? If so, how do we create the fertile conditions for it to increase?
- Can the use of Open Space throughout an Agile adoption increase the likelihood of Agile adoption success?
We are now at a crossroads in the Agile-adoption narrative. We can advance to a new stage, regress to a previous one, or stay where we are. Which way is it going to go?
Ultimately, it is the customer who determines what is valuable … and what is not.
Will the organizations that consume Agile coaching start asking these questions? Or will they remain happy with the results they are currently getting by imposing Agile on teams? The answer to these questions will determine how the Agile narrative unfolds over the next several years.
About the Author
Daniel Mezick is a management consultant, author and keynote speaker. He is the formulator of Open Agile Adoption, a technique for creating rapid and lasting enterprise agility. He is the author of The Culture Game, a book describing sixteen patterns of group behavior that help make any team smarter. The book is based on five years of experience coaching 119 Agile teams across 25 different organizations. Daniel’s client list includes CapitalOne, INTUIT, Zappos Insights, CIGNA, SEIMENS Healthcare, Harvard University and many smaller enterprises. Learn more and contact Daniel here.