Article: Lean and Agile, Marriage Made in Heaven or Oxymoron?

| by Amr Elssamadisy Follow 0 Followers on Feb 13, 2009. Estimated reading time: 1 minute |

Dave West takes a look at the world views of the Agile and Lean communities and finds them in conflict. 

Lean views software development as a process for moving from conception to product. It wants to optimize that process, albeit in a radically different way and with radically different values than traditional (e.g. Taylorism) attempts at optimization.

Agile views software development as a process for building a consensual theory of the world: with an artifact being a byproduct - an expression - of that theory.

Because the fundamental worldviews of the two sides are dramatically different, it is inevitable that there will be conflicts. These conflicts will usually manifest themselves at the level of tools and practices.

If true, then many of us in the community blending Lean and Agile and unaware of the inherent clash in ideals could be making some big mistakes.  As an example of a manifestation of this conflict Dave takes the backlog:

Lean looks at the product backlog and sees 'inventory' and 'waste.' Mary Poppendieck is on record suggesting that the product backlog should be eliminated or, at minimum pared to a size more evenly matched to the collective velocity of the teams'.

Agile sees the product backlog as a snapshot view of an emerging theory. Even if that snapshot view is physically manifest as a wall full of story cards, it is not an inventory! The cards on the wall serve as a form of external memory, with each card evoking (recalling to mind) detailed conversations and understandings of how things work.

This article is a must-read if you've been dabbling in both the Lean and Agile worlds.

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Yep! Backlogs are essential. by Amr Elssamadisy

Agile sees the product backlog as a snapshot view of an emerging theory. Even if that snapshot view is physically manifest as a wall full of story cards, it is not an inventory! The cards on the wall serve as a form of external memory, with each card evoking (recalling to mind) detailed conversations and understandings of how things work.

I have never quite looked at a backlog in this manner before - but it rings TRUE. I don't know about others, but I've been really uneasy ever since it became cool to diss backlogs.

Lean - Misunderstood by Paul Beckford

Lean has become fashionable amongst the Agile crowd that like to think of themselves as being on the bleeding edge. My observation is that many of the Lean practices are misunderstood and applied out of context.

Take waste for example. What is a wasteful activity on a production floor is very different then what is wasteful in a creative new product development environment. I have heard people talk about retrospectives as waste. Taiichi Ohno surely wouldn't agree.

The bottom line is that Toyota came up with the Toyota Production System, by continuously asking tough questions, and being determined and disciplined in their efforts to streamline and improve motor vehicle manufacture. Taiichi Ohno's number one tool was "Five Whys", repeatedly questioning practices to get to the root cause of problems and to gain new insights.

Applying the lessons they learned out of context is not to understand the TPS philosophy at all. The TPS took Toyota over three decades to perfect and they are still improving and challenging assumptions. Good old fashioned hard work and graft may be unfashionable, but Lean is no silver bullet and its not a free lunch either. We still have a long way to go in discovering what lean principles and practices if any make sense in the context of software development. It is too early to say because we haven't done the work yet!


Law of Software Process by Michael Hunger

Regarding the nature of Software Development - The approach that Phillip G. Armour takes in Laws of Software Process - i.e. that Software is the fifth kind of storing knowledge (and make it executable by the way) is also a quite interesting model.

I think there is no one right or wrong you always have to see it in the context. So if a principle or practice helps to improve your efficiency it is no waste. And backlogs do as the knowledge they assemble would have to extracted again and again from the customer.


Tangible, viewable, and negotiable by Mouneer Rabie

Although we have not been with agility for so long, but every team member can attest to the value of the viewable, tangible and negotiable product backlog. By no means, the backlog can be considered waste; after all and so far, it has been “working” for us.
Totally agree with Michael “if a principle or practice helps to improve your efficiency it is no waste”.


Lean and Agile: should cousins marry? by Steve Freeman

Once the dust has settled and the camps really understand each other, I'm hoping that we'll find that "Lean" and "Agile" are too close to formally join. My guess is that the large product backlog and a proportion of the coding belong in the equivalent of the Toyota Product Development System, rather than their Product System.

More in a
post on my blog.

Lean is more than production by Russell Healy

It's "Lean" not "Lean Production" for a reason. Lean is about working to reduce the time and energy spent that does not add value to our customer. It's the mantra of Scrum - "inspect and adapt" - continually look at what you're doing, and try to see if there is a better way.

This approach is valuable for, but not limited to production processes. It also applies to new product development, and therefore to software development. Cross-functional teams and emergent design are examples of applying lean philosophy to new product development.

An aspect of Lean altogether missed in this article, is that of knowledge creation and sharing, which brings the supposed world views of Agile and Lean a whole lot closer. The author and readers of this article are recommended to read a little wider on Lean. Mary's list of recommended reading is probably a good place to start:

Re: Lean is more than production by Paul Beckford

I think the author acknowledges, that Agile and Lean principles and practices were created by applying a common philosophy of inspect and adapt. The point being made though is that although the philosophy and values are very similar, the principles and practices are somewhat different. This make sense to me since the two things emerged from different contexts so you wouldn't expect them to be identical. Agile came from Smalltalk and the challenge of Object Orientated programming, and Lean as described in Mary's first book has emerged from manufacturing, and more specifically Toyota and their goal of becoming world leaders at automobile production.

Based on what is in Mary's first book, I don't think she would disagree with what has been said here, but I don't know what she has said since. I also know that Taiichi Ohno also spoke about the pitfalls of blindly following principles and practices out of context. Which is why he chose not to blindly follow the American approach of mass-manufacture which was highly successful in his time.


more pointless & damaging misrepresentation by rob bowley

Why are some people so determined to be divisive? I listened to Dan Jones (wrote The Machine That Changed the World and formulated the concept of Lean from working with Toyota) talk at XPDay last year. He said saw no real difference between Agile and Lean so why do we? Why pick up on one thing that Mary P said about backlogs? She may have written a great book but that does not make her a Caliph or Umma of the practices ergo everyone must obey her every word. If you read The Toyota Way you'll see how frustrated Toyota get by people focusing on particular practices and concepts rather than making the effort to understand the fundamental underlying principles.

As I said recently, Lean will fail for the same reasons Scrum projects do, because people fail to understand the principles and focus on the practices as Dave is doing here.

Misleading by Sebastian Kübeck

Honestly, I completely disagree your statement. If you had done just a little bit of research you would have known that Agile methods have been highly influenced by the TPS right from the beginning. The Poppendiecks were not the first that introduced Lean methods to software development.


Re: Lean and Agile: should cousins marry? by Paul Beckford

Hi Steve,
I think your point about the Toyota Product Development System being more relevant to Software Development than TPS is a good one. I haven't read any primary sources on this, although Mary does mention some of the ideas you refer to in your blog post in her book.

You are probably right, that if you dig deep enough that Lean as applied to new product development rather than production is very similar to Agile, but as you know most people don't bother to dig that deep and just go on the buzzwords and what they've heard second hand from industry leaders.


Re: Lean and Agile: should cousins marry? by Paul Beckford

As an example of what I mean, I read somewhere that Toyota sent a team of Engineers to california for six months to live the high life to get an idea of what Americans saw as luxury living. This was seen as useful background for the development of the Lexus.

Now Engineers living it up in foreign climes sure sound like an ideal example of waste to me :) Yet Toyota saw it as valuable. Should software engineers all be doing the same too? Well I guess it depends on the context. It comes down to giving up on the idea of lifting "best practice" from others and being prepared to do the hard work to find what works in your context for yourself.


Re: more pointless & damaging misrepresentation by Andrew Walker

Good points Rob. In fact, I would go a step further and say that both lean thinking and agility are far more about state of mind, value systems and establishing a healthy culture. To revisit a point made earlier in the thread, reading about Taiichi Ohno's experiences when he was considering production methods at Toyota, all his ideas were based around open minded thinking - which I find is not such a popular choice these days.

Stepping out of the problem and thinking about it rather than looking for a 'paint by numbers' solution can yield far greater successes - the inspect and adapt mantra cannot be realized without thinking. Far too many ideas are prescriptive - if it doesn't work, change it - so long as values are aligned with success, better ideas can supplant older ones and lead to greater success.

Re: more pointless & damaging misrepresentation by Paul Beckford

Hi Andrew,

I agree with everything you say. The one thing I would add though is that Toyota developed the TPS over 3 decades, and despite senior management buy-in, they still made mistakes along the way and had to deal with internal and external resistance as they went. I doubt the software industry has that patience never mind the determination and discipline.

True cultural change is very difficult, and some people are looking for a quick fix. I read Daves blog post as a cautionary warning that blindly adopting Lean practices isn't it.

Going further, If we choose to accept our western organisational culture as part of our context, that is unlikely to change any time soon, then perhaps we need to go about things differently and find a new way, rather than blindly following the Toyota way. Like you say, we need to keep an open mind.


Things turning a little too negative? by Amr Elssamadisy

It is always good to see a thought-provoking article. The fact that it seems to have caused some conflict means that Dave has touched on something real. I'm not sure that the points that people make about 'misleading, pointless, and damaging misrepresentation' are quite fair.

As I read it:

In a nutshell: Lean and Agile are grounded in fundamentally different world-views and therefore will inevitably find themselves in opposition on critical points.

The fact that Lean (Production) and Agile (Software Development) have different world views does not mean they intersect. Nor does it mean they are at war. They just mean that they do not fully coincide. (Which I think is fair.)


In the following paragraphs I will try to show the opposing world-views, illustrate one point of conflict, and then suggest how the two viewpoints might be reconciled.

You'll see that Dave suggests how the world views can be reconciled.

So, instead of sniping, what about addressing the issue - do they have different world views or not? If not, do you think the differences can be ignored or worked with?

Peace. Amr

Re: Things turning a little too negative? by dave west

Amr, thanks for your comments. If I may amplify:

  • Lean values and principles and Agile values and principles are fully complementary.

  • The Lean world view - the lens through which Lean values and principles are viewed and interpreted - is grounded in process, specifically production process. This same interpretation is applied to specific practices and specific metrics.

  • The Agile world view (excluding, for the moment, Scrum) is grounded in "theory building" and specific practices are interpreted through that lens.

  • At the level of values and principles there is little conflict - but, because of the world view lenses, similar values lead to the advocacy of different practices, and occasionally those practices are in apparent conflict.

  • My discussion of the product backlog is one example of such conflict.

  • not discussed in the article, but Scrum is often misinterpreted in a similar way - as if it was nothing more than an alternate means for imposing Taylorist management principles on software development.

The fact of conflict would be hard to deny - the question of how to resolve that conflict is at issue here. I would assert that it can only be resolved at the level of values and principles. And to accomplish this we need to be consciously aware of how world views (cultures) affect our interpretation and ultimately our expression of those values in concrete practices.

Like many of you who replied to the article - my ultimate goal is to advocate for a thoughtful, fully informed, and deeply reflective software development and the avoidance of doctrinaire, knee-jerk imposition of "by the book" practices.

Re: more pointless & damaging misrepresentation by Jeff Santini

I don't think you can apply the adjective blind to "follow the Toyota Way". If part of the way is to examine what you do and adapt, then what pls give an example of how to do that blindly.


Re: Things turning a little too negative? by Jeff Santini

May I suggest that it is not that hard to deny the conflict, when what was presented was highly hypothetical, and I have never seen any real conflict myself.

Your suggested solution to the conflict implies that both Lean and Agile are self-contradictory. If the solution is as you state -
"Both Lean and Agile must stop applying, in a literal and rote manner, the tools and practices." and both Agile and Lean are in favor of inpecting and adapting, as their guiding principles suggest that they should, then they cannot be applying "the tools and practices" in a "literal and rote manner" unless they are not following their own principles.

I can understand the differences highlighted in the article, Although I think event these hypothetical differences require a narrow interpretation of each Agile and Lean, but I think to argue that these differences are causing real problems in software development environments today requires a little more evidence before the reader can be asked to accept the argument given.


Isn't a huge backlog a truly waste? by Guilherme Lopes

I think product backlog was a bad example to make your point. Actually I don't see product backlog as a main practice of agile methodologies.

I already worked at several projects with huge backlog lists that worth noting. It's even painful to remember when Scrum Master insists in review all those cards (that God know when one of those will come into current sprint) every meeting regardless the bored team views no value on that. The client asking us to remove half of those cards because they don't make sense anymore was only a matter of time.

Mary and Tom made their point quite clear when they said that there are differences between Lean Production and Lean Software Development. The key was the principles, tools and process can be adapted.

Guilherme Lopes

Re: Isn't a huge backlog a truly waste? by Amr Elssamadisy

Actually I don't see product backlog as a main practice of agile methodologies.

So what mechanism(s) do you use to understand the problem at hand and have a conversation with your customers? Do you keep a record?

Discuss programming as theory building by Jason Catena

I recently published a "survey" which presents the "Programming as Theory Building" paper (Naur 1985) and later comments from papers and web. It includes the relevant excerpt from Cockburn 2006.

Abstract: The present survey republishes major articles, reprints leading discussion, and collects erudite comment on the theory-building view of software programming.

Just not practical by Jack Milunsky

Whenever I read blogs like this I often wonder who the target audience is. From my perspective, this is way to theoretical for anyone in the trenches to get any real benefit from this argument. My sense is this is just a way to stir up conversation amongst the purists. I don't believe anyone doing actual software development really cares - sorry. My opinion is that Lean has some great concepts that really work well with Agile methods and I have written about this on my own blog at

This was way over the top for me
My 2 cents

Re: Isn't a huge backlog a truly waste? by Guilherme Lopes

So what mechanism(s) do you use to understand the problem at hand and have a conversation with your customers? Do you keep a record?

We all (client and developers) have a shared vision about the product. We discuss it several times in week. The client is really close to us, actually We have lunch together every day.

But definitions, written on ink and paper, issue tracker or whatever, are only made for the very next iteration.

We don't need to keep a record. The important features will keeps coming back, naturally.

Valentine day anyone? by Olivier Gourment

Can you see by the number of comments just how bored software professionals get around Valentine day? They will do anything, even post very long comments, to flee their romantic duties.


Just wanted to make a point here, that we have ALL made this mistake. There is no ONE "Lean". The TPS (Toyota Production Systems) and the TPDS (Toyota Product Development System) are TWO entirely distinct systems. Just to cite Jeffrey Liker: "Most Toyota Product Development managers claimed they had very limited knowledge of TPS, and Toyota engineers did not see TPS as the launching point for lean processes in product development."
And it should be noted that the Poppendiecks published a second book, that underlined the same mistake at its very beginning. If someone is considering reading about Lean Software Development, get the latest book (2007)!

Make love, not war.

Lean does not wear “production glasses" - people do. by Jay Packlick

Large queues in systems exhibit certain characteristics (see Little’s law for example). The optimal size for a given queue depends entirely on context so an assertion that a large backlog (a form of queue) is either good or bad absent of a project context is fallacious. In general cost and average time to produce value increases as a queue grows however; if the net value created for the customer is increased with a larger queue then a larger queue is preferable. In other words, big backlogs are bad except when they’re not. The singular goal of lean is to increase value. Fixating on one principle or constraint (i.e. cost increases with queue size) without balancing competing constraints (i.e. more design variation / theories can increase value) is a misapplication of lean.

Despite its origins, the core principles of lean thinking are orthogonal to the specific practices that have evolved. Lean principles gave rise to lean production; they’ve also given rise to lean development practices. In production variation usually decreases value. In development, variation often increases value. ‘Lean’ does not wear “production glasses”; far too often the people trying to apply lean principles and practices do.

If we’re going to advance the state of the art and help our teams we need to focus less on practices and more on the underlying principles. Lean development is no more about ‘small backlogs’ than Agile is about ‘no documentation’. Lean and agile principles are entirely complimentary. Often, the practices evolved are not. What we need are more practitioners and coaches in the field who deeply understand the underlying principles of lean and agile, who live and breathe ‘innovate, inspect, and adapt’, and aren’t wedded to single-source methodology dogma. That might not sell as many books, but it sure as heck will increase value for our customers.

I greatly admire and appreciate the Poppendieck's contribution to improving software development but to gain a fundamental understanding of Lean you’re going to have to read more deeply and broadly than their two lean books. Don Reinertsen has an illuminating new book coming out in May (The Principles of Product Development Flow: Second Generation Lean Product Development) that I strongly encourage anybody trying to get a handle on applying lean in a development context to read.

Jay Packlick

Programming as Theory Building by Mary Poppendieck

I like the idea of Programming as Theory Building.

In lean, learning starts with building a theory and then testing it, finding where the theory does not hold up, modifying it, and so on forever. It's pretty much a constant Plan-Do-Check-Act cycle. You do this with the product (software). You do this with the development process (constant improvement).

So I suggest that the way to handle a backlog is to build a theory on how long it should be in your environment, create a theory on what the results of that length would be. Devise a way to test the theory. Run the experiments. Learn.

Mary Poppendieck

Backlogs don't necessarily serve effectively as a theory of the product by Jason Yip

Backlogs as officially described are just a list of items. These flat backlogs, as Jeff Patton calls them, are not really that effective as models for the theory of the product or system. In essence, they are just a list of items to do.

A story map, a Parking Lot diagram, process map, etc. serve the role of theory model better than a backlog.

The other important point, which is really only apparent when we look at the detailed day-to-day issues of software development (and even product development), is that there is still a significant part of what we do that is more repetitive than not. Especially when we realise that most software development is maintenance, not new concept development.

Of course, software development is not the same as what you see in any particular factory. But that's why the essential part "Lean" is not the particular solutions designed for a particular context. The essential part is the worldview: respect for people, continuous improvement, problem solving, focus on the customer, etc. It's a very common message in the Lean community about not just inserting some off-the-shelf solution into your own situation but actually thinking about what's going on.

Someone with a very tool-focused worldview will believe that Lean and Agile are incompatible. Someone with a values-focused worldview will see that the thinking approach is very compatible. I'd even argue that people with a tool-focused worldview don't really get what was eventually called Agile anyway.

Re: Backlogs don't necessarily serve effectively as a theory of the product by dave west

I really like this metaphor - enough to quote it here<blockquoten my head a see a tree where the trunk is built from the goals or desired benefits that drive the system; big branches are users; the small branches and twigs are the capabilities they need; then finally the leaves are the user stories small enough to place into development iterations."

"After all that work, after establishing all that shared understanding I feel like we pull all the leaves off the tree and load them into a leaf bag - then cut down the tree."

"That's what a flat backlog is to me. A bag of context-free mulch.">I agree with the sentiment and agree that the "flat product backlog" is not the best tool for advancing theory. Fortunately, the backlog is not the only agile practice that promotes theory building - stand-up meetings, big visible charts, automated tests, collective code ownership, wall-to-wall whiteboards, retrospectives, planning games, etc. make their contributions. In concert they may or may not be enough to fully support theory building and any one of them could certainly be improved.

Unfortunately, very few people look at these practices and see their contribution to theory building - just as very few see their relation to the underlying values and principles of agile. The most controversial accusation I made in the original article was implicit: practitioners of agile, and lean, do not really understand WHY they are doing what they are doing. Because of that they tend to misuse, misinterpret, or misapply the practices.

A product backlog is one example. As a coach, not a theorist or academic, I have very rarely seen teams using the product backlog as anything other than an inventory of to do items. (Until I coached them differently, of course. [smile]) If this is the extent of your understanding of why a backlog exists, it will be hard not to see it as waste.

A similar error occurs with either and both of the Toyota models - the superficial understanding of the practices - i.e. an understanding that is not deeply grounded in an understanding of the underlying principles and values - leads to the same misinterpretations. In the case of Lean, it is really easy to make the mistake of interpreting the practices as being solely related to the idea of production - because the production model is the de facto culture of software development.



Re: Backlogs don't necessarily serve effectively as a theory of the product by dave west

sorry about the formatting errors in the previous reply - the system gave me an error message when trying to preview and then posted when I acknowledged the error - not expected behavior.

Re: Lean does not wear “production glasses by Arun Batchu

Very well said - I have very little to add after reading your comment. And, thanks for the reference to the upcoming book.
Most of us have accepted the Poppendieck's books as the launchpads of our journey and moved on. Theory building, without a doubt, happens all the time when you are doing Lean properly. Lean is all in the "thinking" - in fact some of the best books about Lean are those address the thinking - achieving deeper undrestanding, model building - theory building.

if you read a little bit more literature about the "other" TPS - Toyota Product-development System... you will find that the theory is in violent agreement with what you have to say.

A wonderful book that might help address some of the misgivings of Lean as applied to software development is : . It is one of the most "highlighted" of all lean books I have in my stack.

Re: Yep! Backlogs are essential. by Lisa Crispin

I'm not following why backlogs are essential. We were convinced (partly by Mary and Tom) that our backlog was silly overhead. We only keep enough in it now for the next couple or three sprints and this has been working fine for a year or two. I feel that we have adjusted our "agile" with some "lean" principles and it's working for us.

Re: Backlogs don't necessarily serve effectively as a theory of the product by Richard Karpinski

These stories in the bscklog are isomorphic with the long list of requirements in formal IT contracts. When there are six or ten of them that all the customers are agreed on, this will work well. When there are fifty or two hundred, then they are likely just collected lists of wishes which have not been vetted by the other customers.

Don't accept them. Treat them as hints and use the five (or more) whys to find the business purpose of each. Take the five or ten ultimate business purposes as your goals and decide how to quantify progress toward each goal. These might include saving time or other resource, increasing revenue, decreasing complaints, or any rational business need.

Now engineer some approaches toward satisfying each goal. Discuss these with the product owner or better, the customers. Build them into stories. Estimate the impact that each story will have on each goal and choose the greatest impact lowest effort stories to address first.

Everyone will be happier with the results and every implemented feature will be used. And you can quit when the expected progress toward the goals of any remaining story is worth less than the cost of implementing it.

Re: Misleading by Paul Oldfield

In a forum debate that included several of the original members of the agile alliance, they agreed that lean thinking had not been an explicit influence on their thinking at the time of the manifesto. Sure, we can look back today and see the parallels and the synergy, but let's not be revisionist.

Re: Valentine day anyone? by Randy Hough

I had an awesome Valentine's Day with my sweetie in Quebec City!
On the practical front though, our Lean Initiative seems to have slowed to some kind of crawl speed. As one of the outsider observers at a major plastics manufacturing company I get to see all manner of programs started, revved up, a great deal of activity moving, and then.... it typically slows down due to some practical concern. Nothing moves in a straight line, especially Lean manufacturing.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

33 Discuss