Large-Scale Agile Design & Architecture: Ways of Working

During my 2011 QCon London keynote on "Scaling Lean & Agile: Large, Multisite or Offshore Delivery", I mentioned — as an aside — that, "Architecture is a bad metaphor. We don't construct our software like a building, we grow it like a garden." This prompted many a tweet, and some people were interested in clarification or elaboration.

Weʼve included a free PDF of our book chapter "Design & Architecture" as a download to accompany this article.

The first thing Iʼd like to emphasize is that this is a very old point and metaphor. I started work programming in the 1970s (as an APL programmer), when Harlan Mills was an important thought leader. Many programmers back then, including me, were influenced by his writings, which emphasized top-down programming; for example, the paper, "Top- Down Programming in Large Systems" [Mills 1971]. Mills talked about incrementally growing systems from thinner to thicker vertical slices of functionality with step-wise programming refinement. Thatʼs how I learned to design and evolve large software systems, starting in the insurance industry.

Related Sponsored Content

Related Sponsor

• June 22, 2023, 10AM PT

  More Wood & Less Arrows: How to Build an Efficient Cloud Application Architecture

  Presented by: Sean Chittenden - Director of Engineering

Then, in 1986 (often attributed to 1987, due to a re-publishing in IEEE Computer), Fred Brooks published the famous article, "No Silver Bullet: Essence and Accidents of Software Engineering" [Brooks 1986]. As with many other programmers in the 1980s, this also had an impact on me. Some quotes from No Silver Bullet are noteworthy, with respect to the growing/gardening metaphor:

Incremental development—grow, not build, software. ... the building metaphor has outlived its usefulness. It is time to change again. If, as I believe, the conceptual structures we construct today are too complicated to be accurately specified in advance, and too complex to be built faultlessly, then we must take a radically different approach.

Let us turn to nature and study complexity in living things, instead of just the dead works of man. Here we find constructs whose complexities thrill us with awe. The brain alone is intricate beyond mapping, powerful beyond imitation, rich in diversity, self-protecting, and self-renewing. The secret is that it is grown, not built.

So it must be with our software systems. Some years ago Harlan Mills proposed that any software system should be grown by incremental development.

So, I grew up with the metaphor of growing rather than building software systems, and when I first heard the term "architect" in software development (sometime in the mid 1980s) I still remember my feeling of cognitive dissonance. It struck me then as an odd name and metaphor, not consistent with my and my colleagueʼs experience of developing systems, large and small. And in the 1970s we were very familiar with the dysfunction of the kind of "architects" that today are called "PowerPoint architects" who are not hands-on master programmers, but who were far away from the reality of the code.

Consequently, to say in a keynote in 2011, "Architecture is a bad metaphor. We don't construct our software like a building, we grow it like a garden," is to share the default view that I and many colleagues from the 1970s had and have — it certainly wasnʼt meant to suggest some novel ʻagileʼ design insight or fringe idea.

And this is an oft-told message; for example, in The Pragmatic Programmer: From Journeyman to Master [HT 1999] — a personal favorite — Andy Hunt and Dave Thomas repeat, "Rather than construction, programming is more like gardening." They are once again emphasizing the growing, living dynamics of software development, very different than the building, "engineering" metaphors that have confused our discipline.

Well, "So what?", you ask; itʼs just a metaphor. (And by the way, "gardening" is just a metaphor with strengths and weaknesses too.) Well, itʼs a very interesting time for me to be writing this short article, because just two days ago, I was sent an email from someone asking about how to do accounting in the context of iterative and incremental concurrent-engineering development (e.g., in Scrum). For example, what could be allocated to operating expense? What could be allocated to capital expense (which has different depreciation and tax implications).

You see, a word such as "architecture" is not just a metaphor in its impact; it has concrete influence on policy and people. To pick up the accounting opex/capex story again, it turns out that accountants in the USA have looked at the "phases" and activities of software development through their naive view, to answer their question of activity- based accounting. These accountants learned (read, heard) that there is an "architecture phase" done by "architects", followed by a "construction phase" called "programming." Just like a concrete building, they have reasoned by analogy inspired by the words they are hearing (since they donʼt really know whatʼs going on), that the creative design step is the "architecting" done by the "architects" (and itʼs opex), and the "programming phase" is a non-creative manufacturing activity like making doughnuts on an assembly line, from the "architectural software blueprint." And itʼs capex.

Of course, this is utter nonsense. Any really experienced software developer knows that.

But the point is that words can have strong influence in shaping mental models, and then policies and behaviors. A non-software-expert with policy power (e.g., a CEO, a VP of HR or Finance, an accountant, a lawyer, ...) hears the word "architecture", and makes naive assumptions that impact policy, process, interactions, career paths, and much more. Itʼs not just a metaphor after that.

Therefore, in our latest book, Practices for Scaling Lean & Agile Development: Large, Multisite, and Offshore Product Development with Large-Scale Scrum, [LV 2010] Bas Vodde and I wrote the "Design & Architecture" chapter that starts with the suggestion, Try...Think ʻgardeningʼ over ʻarchitectingʼ—Create a culture of living, growing design, which we elaborate in seven pages of more detailed explanation. And thatʼs followed by about twenty other suggestions for ways of working to support large-scale agile design and architecture.

You see, we believe that agile architecture comes from the behavior of agile architecting. Itʼs primarily about mindset and actions, not the use of a particular design pattern. And part of that mindset is thinking ʻgrowingʼ or ʻgardeningʼ over ʻarchitectingʼ.

For those interested in further exploring "software gardening" and our other tips for the behavior of large-scale agile architecting, weʼve included a free PDF of our book chapter "Design & Architecture" as a download to accompany this article. Other good readings on "software gardening" include the books and on-line writings of the Pragmatic Programmers: Andy Hunt and Dave Thomas.

References

[Brooks 1986] Brooks, Jr., F.P., "No Silver Bullet—Essence and Accidents of Software Engineering," Information Processing 86. H.J. Kugler, ed. Elsevier Science Publishers B.V. (North Holland)

[HT 1999] Hund, A., Thomas, D. The Pragmatic Programmer: From Journeyman to Master. Addison-Wesley.

[LV 2010] Larman, C., Vodde, B. Practices for Scaling Lean & Agile Development: Large, Multisite, and Offshore Product Development with Large-Scale Scrum. Addison- Wesley.

[Mills 1971] H.D. Mills, "Top-Down Programming in Large Systems," in Debugging Techniques in Large Systems, R. Ruskin, ed., Prentice-Hall.

Authors Backgrounds

Craig Larman is a management and organizational-design consultant for enterprise adoptions and large-scale product delivery with agile principles and Scrum. He is the co- author (with Bas Vodde) of Scaling Lean & Agile Development: Thinking & Organizational Tools, and Practices for Scaling Lean & Agile Development: Large, Multisite, & Offshore Product Development with Large-Scale Scrum, and author of Agile & Iterative Development: A Managerʼs Guide. Craig has coached management and delivery groups at, for example, Alcatel-Lucent Networks, Xerox, Nokia Siemens Networks, Computer Associates, and UBS. He has also introduced agile delivery in the USA defense industry. Craig has served as chief scientist at Valtech, a consulting and outsourcing company with a division in Bangalore, where he and colleagues created "agile offshore development."

Bas Vodde is a consultant/coach with focus on organizational and technical adoptions of agile concepts. He is the co-author (with Craig Larman) of Scaling Lean & Agile Development: Thinking & Organizational Tools, and Practices for Scaling Lean & Agile Development: Large, Multisite, & Offshore Product Development with Large-Scale Scrum. Bas spent several years leading the Agile change program at Nokia Siemens Networks in Finland but now works for an agile coaching company he founded in Singapore with offices in China and Japan.

Craig and Bas are also the authors of the free, online Lean Primer.