Does Value Stream Mapping Work for Software Development?

Remember that everything we have in our arsenal is just a tool no more and no less. Sometimes a tool is applicable and sometimes it isn't. I've used Value Stream Mapping with one group of software executives to help them see waste in a bug fix cycle, from a customer reporting a bug to shipping them a fix. The process wasn't entirely linear but none the less value stream mapping helped them discover alot of waste. I think the key in this case (and many others) was to follow the life of one bug through the system. It wasn't a real bug, but as an example it made the whole process more real.

Like any other tool it has it limitations, i.e. around cyclic activities. So its a great tool in your arsenal just remember it isn't always a good choice.

Cheers
Mark Levison
Agile Pain Relief Consulting

Hi Mark,

>> So its a great tool in your arsenal just remember it isn't always a good choice.

I agree, the problem though is a lot of this stuff is "sold' as a silver bullet, so I wouldn't describe pointing out were some of it doesn't work as "twaddle". I think the Lean/Agile community need to be less defensive and accept that they don't always have canned answers. In the early days the Agile movement was more frank about this. Since Agile/Lean has become a consulting gig the message has been we've got the answer to everything. Big teams, geographically dispersed, old style management, it doesn't matter just use our snake oil and all your problems will be solved.

Given that all practices are just tools, I think we should spend more time discussing the contexts where each practice is appropriate rather then just presenting them as though they were universal solutions. That way people stand half a chance of choosing the right tool for the job.

Interestingly value stream mapping worked well for you for bug fixing, which can be considered a relatively linear, repetitive industrial process. At the other extreme is a project charter like "create me an iPhone beater", here the job at hand is much more artful. These two problems call for a different approach.

The best book I've seen explain this is Artful Making by Austin and Devin. It places Agile and Lean ideas in a wider historical context and explains where, when and why artful making versus industrial making is more appropriate:

www.amazon.com/Artful-Making-Managers-About-Art...

I believe we should spend more time discussing what works, where, when and why and placing these practices in a wider context. Otherwise we are no different from say RUP; offering a burgeoning toolbox with no instruction manual.

I would agree with Paul. Last month while doing a software craftsmanship and TDD training, I was amazed to notice the complacency of the team just because they had done a so called "value stream mapping". According to them they had eradicated all the waste in the system and they had nothing else to improve on.

I think I fully agree with

>> the problem though is a lot of this stuff is "sold' as a silver bullet, so I wouldn't describe pointing out were some of it doesn't work as "twaddle". I think the Lean/Agile community need to be less defensive and accept that they don't always have canned answers.

and it does make sense to talk about situations where certain tools work and others don't. Right now, with that team at least, the way Lean had been "sold" to them was that you do VSM and you have conquered the world.

First of all it's a tool, and you have to use it correctly. For example you have to regard the whole system, from idea till release.

You'd be amazed how much waiting time (only 1 of the waste-types) is spended (or better waisted) in 95% projects/firms/engagements.

Paul

Could not agree more.

The "Silver Bullet" problem is huge. Sales people are selling "Agile" and "Lean", often without even understanding them. "Thought Leaders" are selling their ideas as the ultimate solution. People then buy them and fail, which gives the ideas a bad name and make it harder for those who follow in their path who can apply them.

In the past when these ideas were not fashionable the people trying them were much more cautious and considered context to be king. These people understand the context where these tools worked and more importantly, when they did not.

Unfortunately Lean and Agile are now dominated by people selling the ideas. From my experience, some of them cannot do the things they sell in such a polished manner. It is also the case that the Sales Machine does not want people to hear the message that some of these ideas sometimes do not work or are limited in their effectiveness. So through simple disinterest, and by ignoring them, or outright hostility, the Sales Machine of Agile and Lean have managed to create a community that is of no interest to the experienced practitioners. Until this group is bought back in, things will go from bad to worse.

As a salesman and their tool allways works.

As a practitioner and they will say it sometimes works.

Paul, I used twaddle because it seemed both Jurgen and Tobias were throwing the baby out with bathwater. In addition you can use Value Stream Mapping in situation with loops. Just follow the progress of one item through the system. It won't help you discover all your problems but it will help you visualize a bunch. Even the bug fix process I describes had loops. None the less a map was valuable. When we see the tool mis applied we shouldn't dump on the tool. Instead we should help people see how they misused it.

Cheers
Mark

Hi Mark,

I read something else in what they had to say. Most people aren't aware that Lean manufacturing is an evolution of Taylorist thinking and scientific management. In fact Taiichi Ohno makes it clear that he took his inspiration from Henry Ford. Now I'm not trying to throw Lean under the bus :), but knowing this it is easy to draw parallels between value stream mapping and say a time and motion study. Whilst an approach like "time and motion" may be well suited to industrial work, it is not well suited to knowledge work.

So we need to be very careful when "borrowing" practices from Manufacturing. Software development is much more people centric, and "human" systems are far more complex then industrial systems. There is a tendency in the software industry to lean towards reductionist ideas. So I see both Tobias and Jurgen challenging a mindset more then a specific practice.

People tend to see the world through a lens of their own choosing. If you are blind to everything other then an industrial metaphor, then you will only see industrial solutions as viable (like speeding up the conveyor belt and removing delays). I think this is the point they were making. There are alternative ways of looking at things that are perhaps less instinctive to some.

Granted, the level of dysfunction in a lot of software organisations means that there are blatant delays and "wastes" that can be addressed just by a simple analysis of flow (or lack there of). Beyond such low hanging fruit however, there is a deeper appreciation of the "system" that is easily missed. I think this is what they were speaking to.

Software doesnt exist in a vacuum. Its all part of some process. If you have a crap process that is codified by the existence of some software, then it doesnt matter if the software is written pooly or well, by improving the software you still have a crap process.
Thats how decent software projects succeed, not by churning out working software, but by going back to basics and allowing the users to redefine the problem and figure out the best way to ease the problem, and writing some software to do it.

Paul - I agree with your basic point that any analogy (not just the manufacturing one) has serious limits. But as quoted by Vikas the statements from both Tobias and Jurgen seem to be very categorical. "By continuing to think in manufacturing metaphors we will continue to bind our thinking to manufacturing practices. Think differently" - this isn't just saying use this tool wisely and understand its limitations. We shouldn't assume we're like manufacturing, on the other hand we don't need to dismiss it as a source of ideas.

I seek ideas and tools from all places, value stream mapping is just one (and a rarely used one at that).

Cheers
Mark Levison
Agile Pain Relief Consulting

Hi Mark,

I seek ideas and tools from all places, value stream mapping is just one (and a rarely used one at that).

I agree James.

Your point is exactly what it is about! Customer Value.

Adding to your comment.
Assuming that the process is ok and needs to be codified, if you look at VSM from a team perspective only then usefulness of VSM is doubtfull. The point is to look beyond your team(s) and look at the whole stream. How is work organized in your department(s) before work even arrives at your team?, What happens after your team delivers?, What's going on when teams are working on it?



Cheers

"Though, there have been several success stories where teams have utilized the value stream concept, however, there are also questions on whether it can be truly applied to software development which is by definition Agile, embraces change and the process itself might undergo a change on the basis of inputs received and the value network generated."

What!? What?! What is that sentence supposed to mean? Where is the editor for this giant run-on sentence, which also appears to be loaded up with sketchy assumptions and punditry. I expect a much higher standard of writing from InfoQ.

> The point is to look beyond your team(s) and look at the whole stream.

I think this is very it becomes quite important and would help a lot. You need to look at the larger picture.

VSM works best to visualise the business processes in organisations. The purpose of Software Development is to support business operations, it's not an end in itself. Therefore, most VSM effort is best spent in visualising the business, as an attention-focusing device for finding ways to remove waste from the way business actually runs.

When it comes to improving the way the Software Engineering work process is carried out, it does help to have a conceptual framework. People must be able to reason about what works well, what doesn't work quite so well, and what experiments might be most worthwhile to try in the next time-box (or retrospective interval if using Kanban). Whether you attempt to use VSM to visualize the software engineering process itself is an interesting choice. The spirit of VSM has merit, even for software development. Some of the specifics of it, less so.

Really, it is just about using VSM correctly.

Creativity has more to do with how all the entire features come together than each feature (or each requirement or change to the product). The job of a product planner (who is responsible for the entire product) is very creative. He has to decide how the pieces fit together. But once the strategy (business and technical)is decided , the scope for creativity for the agreed features are restricted significantly. Some decomposed features are extremely mundane. developing them are just business as usual if you ask the development team.

Moreover, the number of products a company build is usually significantly smaller than the total number of features. When there is a large number, statistical forces takes place and a normal distribution applies based on the central limit theorem. Each feature becomes just a number on a project tracking system, a burn down chart, etc. True, they are prioritised and categorized, but they are managed rather homogeneously using the same set of principles.

I also recognized that not all features are created equal. That is why they need to be categorized. Some features are extremely easy. In a VSM, they are the ones that flow very quickly from start to finish. Some are more difficult and they flow slower. Some are more challenging the flow needs to be augmented with some decision steps or review steps. Nevertheless, you can still define a value stream for them.

So, do not apply VSM for the entire product. So, if you read books or any literature that attempt to apply VSM on the entire waterfall-development or iterative development for the entire product, it is utterly non-sense. But if you see VSM applied on features, stories, user case modules, graphical items, etc. then they make perfect sense.

VSM has also a number of uses. It can be used as a way to gradually and systematically increase the responsibilities of individuals and teams. A novice might be responsible for a particular stage in a VSM (or similarly a state in a Kanban). As knowledge and competency increases, the novice might be made responsible for more stages. This also applies for teams. An offshore partner might be responsible for a particular stage of the VSM and their scope gradually increases as the client becomes assured of their competency. Note that the overall management of VSM and offshore is more complex and need further discussion.

VSM can be used to integrate the work of different teams. I have worked with a product development unit. They had several teams. There wee several teams that perform development from requirements to code. They worked with another team that was responsible for creative the graphics (i.e. icons, gifs, etc.) and the developers need to work with them to cut these graphics into pieces that fit into the UI framework and layout. For some reason, the graphics team was a common resource used by other product units and you need to give them advance notice. Without adequate control, product development was unable to sync with the graphics team thus resulting in development blockages - development could not complete the feature but had to wait for the graphics team. Now, development did not want to give all specifications to the graphics team, but incrementally. A VSM was used to describe the touch points and Kanbans (feature kanban + graphics kanban) was used to visualize progress.

Defining a VSM might not be easy. Every project is different. So, we need to have a rapid way of assembling an VSM. That is why I proposed the use of state cards. This idea is really not new. Define states is part and parcel of Kanban. But what is interesting is that it is possible to building blocks for Kanbans and VSM.

Cheers
-- Pan-Wei Ng

I don't see what's so difficult about describing the process a scrum team follows every time they write software (to answer the example); The description needs merely be coarse-grain enough to encompass the things that are done say every sprint, story and task.

That should give a clear idea of what contributes (e.g. planning, daily meetings) and what might not (needless meetings, creation of certain artifacts that do not promote better products, etc.)

Totally agree. It is all about using them correctly which might differ a whole lot from project to project. Generally two things bother me about Agile discussions these days. One is the Silver Bullet/Cargo Cult approach where the newest tool becomes yet another one to defeat them all. But just as important is the problem of a tendency to discard ideas on a theoretical basis without ever having used them in practice. People will try e.g. VSM on a 1 day training session and think that makes them experts. After failing to use them correctly, on their current project, they discard the idea by stating that software is product discovery and therefore cannot be viewed from a directional perspective.
I have been using VSM for the past 1.5 years on a number of different projects and found them to be extremely valuable.

As stated by Assaf it is all about making them coarse-grain enough to fit the nature of Product Development and the result will become quite different from a typical manufacturing process. If it turns out that granularity in your particular project is so coarse it does not provide any value – by all means discard it, but at least talk to people that have found them useful before doing so. It could be you are just using VSMs the wrong way.

To me value streams are as much a tool for continuous improvement and visualization/understanding of the entire workflow as they are a tool for measuring the time in queue for particular items. All too often we identify the wrong bottlenecks in software development because we only focus on our own particular part of the process. VSMs can help us better understand the entire workflow and combining them with local WIP limits and Cumulative Flow Diagrams can give us early feedback and help us work closer together as a project team (not just development team).

That is not to say they can be used on all projects, but do recognize their value and the improvement opportunities you are missing out on. There might just be a chance that your inability to map the value stream is caused by a huge amount of task switching, lack of division of responsibility and problems understanding the actual workflow.

VSMs have been enormously helpful to me when helping clients visualize and improve their end-to-end process (where Scrum is usually just one part of that chain).

I recently did a keynote about Visualization at Oredev, there is a VSM example on slides 34-38.
blog.crisp.se/henrikkniberg/2010/11/11/12894884...

That case is also described in the first chapter of Mary Poppendieck's latest book "Leading Lean SW development".

That VSM was the tipping point which caused the company to reorganize and go for cross-functional teams instead of silos and handovers, which reduced their product development time from 2 years to 2-3 months on average.

Quote from one of the developers: "This value stream was career-changing for me". He later became an agile coach :o)

But VSMs can of course also be misused, like any tool.

I agree with Mark.
We use value stream mapping and Kanban with very good success.
It is definitely a good tool for some cases, but just should be applied wisely.

Michael Dubakov
www.targetprocess.com

I agree, much of the waste in software projects comes from the disconnects with the business and infrastructure, where delays for sign-offs and reviews is very common. Same within some software processes such as thee defects fixing - there are a number of stages through defect management and in many projects delay occurs at each stage.

lack of effective collaboration is the root cause, but VS~M can identify the disconnects, make them visible and point in the direction of a solution.

I agree with most of what you say. I am not sure you can effectively model the development process, but you can model the interactions between different parts of the process leading to software delivery and identify waste.

For example on my current project part of the software is delivered by a 3rd party and we provide a request for what we want (derived for business requirements gathering).

With the 3rd party I found software delivery to us to be very slow and causing delivery issues for the core project. I examined what the process for getting their software delivered and there were a number of built in delays, Once mapped and and understood, I was able to ensure the gaps were closed and the delays minimised saving about 50% of the elapsed time for when the request was raised to them and the point when a deployment made it available to us. Nothing changed about the process which was commercially and contractually driven, but I was able to ensure that where approvals were needed, the right people were available at the right time and approval was immediate; the usual gap between code delivery and software test and then deploy were also eliminated.

A more agile process would have been better, but at least we got improvement. that helped remove impediments to the core delivery team.

I think there are many similar areas where time is wasted needlessly that can help most projects in most organisations. Well run agile teams in well run agile companies (are there any?) will benefit less.