BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles Laurent Bossavit: Agile Ten Years On

Laurent Bossavit: Agile Ten Years On

This article is part of the Agile Manifesto 10th Anniversary series that is being published on InfoQ.com.

What's all the fuss about this "ten-year anniversary of Agile"? Is it just that we like nice round numbers, or is there something more to it?

The field of computing is still young enough that ten years are a significant time for anything to have been kicking around. Ten years, in our business, is just barely enough for something to have a "history".

How much do software professionals know or care about history? Nowhere near enough. I have only been in this profession for twenty years or so; you may not want to take my word for it.

"After being in the computing business now for more than half a century, one thing worries me more than almost anything else: our lack of a sense of history." Those are the opening words of an essay[1] by Jerry Weinberg, who was around when IBM (founded a century ago) put him in charge of Operating Systems development for Project Mercury (over fifty years ago).

Everyone knows the saying, "those who do not learn from history are condemned to repeat it". (But do we take it to heart?) Weinberg's essay is an updated version of one he wrote twenty years ago, right after the notion of Structured Programming hit its "peak of excitement". Chillingly, his remarks about the shortcomings of the Structured Programming movement remain relevant under merely substituting "Agile" for "Structured Programming".

History matters, if only to avoid being taken in by distortions.

As an example of distortion, consider the "creation myth" we have invented around the origin of Agile's arch-nemesis the big, bad Waterfall. The Waterfall methodology, the story goes, was widely adopted after people tragically misinterpreted a 1970 article by Winston Royce. Page 1 of the article contained the now-famous diagram of a software project modeled as a series of sequential phases. But this was only the first in a series of increasingly complicated diagrams which only slowly revealed what Royce was actually trying to say; and readers - you know how busy people are - tended to not go past page 1 and overlooked Royce's warnings on page 2: "I believe in this concept, but the implementation described above is risky and invites failure".

The problem with this (admittedly seductive) story is that Royce did not originate the sequential-phases model of software development. The careful reader of historical documents, for instance the proceedings[2] of the 1968 NATO conference which founded the discipline of software engineering, will find ample evidence that this was already the prevailing model.

An accurate understanding of history, and a healthy skepticism of appealing myths, are important components in making actual progress.

Today I am often dismayed at how little attention is given even within the Agile community to recording the history of the various ideas that make up its "body of knowledge".

For instance, those who have studied Agile a little bit would know that "Agile is ten years old" is itself a distortion. The roots of Agile go back well beyond the February, 2001 meeting in Snowbird[3]. Both Scrum and Extreme Programming had been around for the best part of a decade already at that time.

The most dangerous distortion of all is the notion, sadly prevalent today, that Agile consists of a handful of competing "brands" of processes: Scrum versus Extreme Programming versus Lean or Kanban. In the interest of pushing this or that brand, the interesting details of our young history are papered over.

Thus, standard presentations of Scrum tacitly encourage the student or reader to believe that "Sprint Retrospectives" are now, always have been, and forever will be a part of Scrum. In fact Retrospectives are a relatively recent addition to Scrum canon, which they entered by way of Extreme Programming, where they became popular thanks to a small number of individuals who became enthusiastic about Retrospectives following the publication of Norm Kerth's book, Project Retrospectives, in 2001. The "normalization" of Retrospectives as a canonical Agile practice can be dated back roughly to 2006.

An accurate history of Agile is hard to reconstruct because of the bottom-up nature of the movement: its expert practitioners tend to act with creditable disregard for canons or central authorities. They pick up ideas because they find them interesting and fruitful; they test those ideas in the laboratories of their own projects, keep what works and discard what doesn't. They are often unconcerned with priority or paternity.

Agile in 2011 is no longer identical with Scrum, or with Extreme Programming, or with Lean Software Development or with any of the "brands" that visibly claim to be the vanguard of Agile. (That, to, me, is the significance of the 2001 meeting, only clear in retrospect: it marks, not the start of the Agile movement, but the beginning of the downfall of these brands.)

However, it is still hard to know exactly what to make of Agile-of-2011. The nice, round ten-year number tempts us into making up stories that this anniversary marks some kind of turning point; that "Agile has arrived", for some value of "arrived".

I am among those who see this as a form of whistling in the dark.

The value in an accurate understanding of history is that we can assess how much has actually been done, and how much still remains to be done.

How much: the Agile movement has succeeded in overturning many previous orthodoxies. It cannot entirely be credited with the demise of Waterfall; this was already well under way at the time of Boehm's Spiral model, introduced in the late 1980's. I am thinking more of the renewed respectability of "developer testing" - back in the 1970's, and I think largely due to Glenford Myers' books on software testing, the notion that "a developer should never test their own code" became entrenched, received wisdom. The practice of test-driven development, originally formulated in the context of Extreme Programming, but applicable to the vast majority of Agile projects, showed that to be a counterproductive overgeneralization.

Similarly, Agile has recast in a new light many long-standing questions about software design, project scheduling, or the management of programmers. This is not the place to go into the details of these upsets; but I do want to suggest that Agile is a different discipline from either of software engineering or project management, not reducible nor fully belonging to either. There is great promise in this discipline, when it finally reaches its full development.

However we must also be honest about how little has really been accomplished. As Weinberg remarks in his essay, most teams today claiming to "do Agile" are only half-heartedly executing a diluted and poorly understood subset of its practices.

It is easy to rail against "half-assed Agile" and "ScrumButs" and give in to the temptation to blame the victims - to hold it against the students that they failed to understand something that was poorly explained, rather than hold the master responsible.

At this point the Agile movement is still just that - a movement, a community. We are doing, I think, fantastically well at putting up conferences and unconferences and workshops and kicking up new ideas and cross-fertilizing each others' thinking about software. But much less so, yet, at settling down and consolidating, at being accomodating to "newbies", and making even a small dent in the huge task of changing what is still, right now, being taught to software engineering students in universities worldwide.

My hope lies in the Agile community's ever surprising capacity to innovate. Specifically, I think that what is needed is a new generation of Agile institutions - not to replace but to work alongside the Agile Alliance and Scrum Alliance and the many smaller but no less necessary groups putting in the hard work of organizing a conference, an unconference or a Coding Dojo. For instance, institutions taking up the challenge of working more effectively with the research and education communites.

Yes, ten is a nice round number, but Agile at ten is still far from rounded out. Here's to the work that lies ahead - to the next ten years of Agile.

About the Author

Laurent Bossavit was an "early adopter" of Agile, stumbling onto Extreme Programming the year before the Snowbird meeting. He was a recipient of the 2006 Gordon Pask award for contributions to Agile practice. He now heads Institut Agile, a privately funded, independent entity whose missions include growing the Agile business ecosystem, creating stronger links between the business and research communities interested in Agile approaches, and providing stronger empirical evidence on the benefits and limitations of Agile practices.


[1] Beyond Agile Programming

[2] The NATO Software Engineering Conferences

[3] Agile@10

Rate this Article

Adoption
Style

BT