Applying Lean Thinking to Software Development
Lean Thinking has been around for a very long time. The basics were laid out in the beginning of the 20th century. Taking a few minor modifications into account, Toyota originally created this system in the mid 70s (back then it was called the Toyota Production System). Lean is also often used in combination with Six Sigma techniques for statistical control and has been widely accepted as a standard in the manufacturing industry. But it is only in the past few years that Lean has gained some popularity in the service industry, such as hospitals, banking and software factories.
What exactly is Lean Thinking?
Some of you may already know what Lean Thinking is all about, but some of you won’t, so let me clarify a few things here. Lean’s major concept is about reducing waste, meaning anything in your production cycle that is not adding value to the customer is considered waste and should therefore be removed from the process.
Now some waste is necessary to keep your business running, such as certain approval cycles, additional quality assurance validations etc. But most of the so-called waste is actual waste of time and effort and should be removed all together. Ideally, you would end up with a 100% waste free production process, but we’re not living in a perfect world, so that’s going to prove to be pretty damn impossible.
Detecting waste in a production environment is relatively easy (if any manufacturing people are reading this, I know that isn’t always the case, but the concept of waste is mush simpler to picture in a manufacturing environment): imagine you have a production process where you create cardboard boxes. To create those boxes, you have to cut away some parts of the cardboard. Those pieces can’t be used in the final product, so they are in fact waste. A solution such as choosing a different box shape or layout may reduce such waste, or eliminate it all together. Looking at a pharmacy company, there is a lot of waste in the approval process of a new medicine, but would you want to take the medication if it wasn’t approved by the FDA or equivalent agency in other countries? That is therefore considered to be necessary waste.
Use the Scrum board
Now, how do you apply these Lean principles in an IT environment? Well, most of the IT guys out there are already using Lean in their daily routines. The widely accepted Scrum board is actually a Kanban, a tool which Scrum has borrowed from Lean. That means that our starting point is that Lean is not a replacement for Scrum (it can be if you want it to), but rather a complementary methodology, because Scrum is created to help control a software development cycle, Lean looks at the entire process and includes more than the development cycle alone. However, you can use Scrum as a tool in any Lean project if you really want to.
Set up a pull system
Another tool from Lean that I’m quite fond of and which has proven to be very effective in my current project is setting up a pull system. In short, a pull system is a system wherein you only start the next task when another task has been finished. This is meant to reduce the amount of Work In Progress (WIP). Little’s Law, which I’ll discuss in detail in just a minute, is a valuable formula to use to determine you optimal WIP count.
So a pull system might seem quite the opposite from Scrum at first, since Scrum advocates committing to a certain number of tasks to be delivered in a sprint. But if you look deeper you will discover that committing those tasks to upper management doesn’t mean you have to put all of them on the board at once! In fact, reducing the amount of tasks at hand will help your team to focus on specific tasks and not run around from one task to another without adding any value and thus adding to the amount of waste (I’ll come back to this later too). And of course you’re team isn’t stressed out from the beginning, seeing the huge amount of work that needs to be done. So there’s a psychological factor in here as well, which should not be underestimated…
Reduce the setup time
In the previous paragraph I already talked briefly about developers running everywhere, not contributing to any tasks but still spending time on them. Well, this is one of the wastes you have to be aware of and it’s called setup time. The best way to describe setup time in an IT development environment is to describe the actions a developer has to take when (re)commencing work on a certain task. First he has to clear his mind from the previous task (yes, this might seem like some Bruce Lee talk, but it is actually true), then he probably needs to start logging his time in some kind of time tracking system, he needs to read up on the new task, might have to pull the code in question from the GIT server, find the proper file and then get cracking at it. That is a lot of work to get set up for the task, which of course takes time, which we appropriately call setup time.
Now, imagine when a developer constantly has to do this because he is asked to “quickly” look at another issue, or stop working on a certain task in favor of starting a new one. That is a lot of time you’re losing in a day! Reducing setup time is one of the most effective measures you can take when applying Lean Thinking in an IT development environment. It’s low hanging fruit, a thing that gives many benefits with the least amount of effort. But of course, the system in place has to work with you. If you have sluggish time tracking systems, or no way to get an overview of what has been done and what needs to be done, that will work against you and will increase the setup time. If that is the case, you’ve already found you first improvement project!
Lean helps you get the job done!
All of these techniques (setup time reduction, pull system etc.) help your team to focus on getting the job done and not getting distracted by things they should not be focusing on. That means that the so-called Process Lead Time or PLT will reduce as well. Process Lead Time is the time a task takes to get through the entire process, from request to production.
Little’s Law says that your process throughput is defined by dividing the amount of work in progress by the process lead time. That means that if you make the process lead time smaller by reducing the waste, you increase the throughput. But this also happens when you reduce the amount of work in progress. That’s exactly where the pull system comes to life.
The 7 wastes of IT
Of course, increased setup time can be caused by a lot of circumstances. After all, constantly switching tasks is not something you’d do naturally, so this is often introduced by some other forms of waste.
In a manufacturing environment, these are considered to be the 7 wastes:
While some of these may sound obvious in your development environment, for others you may have to change the way you look at things to actually see the waste. To help you change your point of view, allow me to clarify how these manufacturing wastes translate to a software factory.
Transport is probably the hardest waste to discover in a software development environment. When you think of an IT department, the end product is a virtual thing. It’s a piece of software that resides on some server. How on earth will this be transported? Copy on a CD/DVD and ship it to the customer? It could be, but that it completely at the end of the line in your process and most of the time, this has nothing to do with de IT department itself. So how exactly can tasks or bug fixes be transported during the development process?
Instead of thinking about a physical transport, you have to think about how tasks get from one developer to the next, from designer to developer, from the analyst to the designer etc. A practical example of transport waste is when a developer has finished a task and does a hand-over to a tester. Let’s assume the tester has just finished another task and the task can be picked up immediately. The tester first has to look at the task to understand what it is supposed to do or fix. Then he has to start up the application and get to the proper step in the application to test what has been programmed. I’m sure you can imagine that it takes some time to get to that point (and I’m still leaving out the possibility that the task needs to be deployed on a test environment first).
This is what is called setup time and in this case it is generated by the hand-over to the next step. Of course, there are many other situations where the setup time comes into play, but this is just one example to demonstrate a point. Another clear example where transport is a waste is the fact that when you hand over a certain task, it usually does not get treated immediately by the next person in the process chain. So, in a way, transport also introduces additional waiting time, which I’ll discuss in detail later on.
Inventory is another form of waste in a software factory that might not seem obvious when you first think about it. It’s software! You don’t stack software somewhere in a warehouse where it can result in overstock, right? Actually, you kind of do that, with the only difference that you call it a backlog. The more items you have waiting to be tackled, the more stock or inventory you’re building.
Now, backlog isn’t the only type of inventory you have in your software development environment. How many items, tickets, feature requests have you begun working on, only to have to put them on hold for a while because you have a higher priority, you are waiting for another piece of software to be installed first, you have to wait for a customer’s response, etc.? Starting a task and not finishing it straight away for whatever reason – the other 6 wastes can be reasons for this to happen – also results in inventory building up.
Motion together with Transport is the hardest waste to discover at first. You really need to start thinking about the tasks and processes differently. When you talk about motion in a software environment, you automatically start thinking about how a task, which is a virtual item, can actually move. But that’s just a wrong point of view.
Motion is all about physical motion: people or objects that are moving. And when they are moving, they are not contributing anything to the Value Added time. However, not all motion can be considered waste. If you look at manufacturing, there motion can easily be identified by people having to get materials in different locations, or the product having to be moved to a storage room or even to the client’s location. Movement is logical and easy to understand in that kind of environment.
But what about motion in a software environment?
One of the motion wastes you might not consider at all is the motion the end user has to perform to work with your software. They may have to perform numerous keystrokes or mouse movements to fulfill a task. For example: just compare what you need to do in Word to get a copy paste between 2 documents when you only use the menus. Don’t you love those little icons, or even better: the shortcut keys?
Also, people don’t sit at their desk all day long (ok, some of you might think so, specifically for the software industry). But the truth is that people are actually running around in the office quite a bit. A couple of examples:
- Getting and updating tasks on the Scrum board
- Unavoidable meetings
- Talking to other developers/testers/managers
You would be amazed at the distance your team members are actually walking each and every day. Now, how do you eliminate the waste of motion as much as possible? Well, you probably know this instinctively, but putting people that are working on the same project in the same room works more effectively than when they are separated by a wall, floor or sometimes even another building. Why is that? One reason is simply that communication is a big factor in performance optimization, and with shorter communication lines (literally) communication comes more natural, more direct and more instantaneous. Which brings us automatically to our next waste: Waiting
When Work In Progress (WIP) sits around, waiting for the next step in the process, it is not being handled efficiently. Tasks that are waiting on something or someone are adding Non-Value Added (NVA) time to your process, delaying the delivery of not only that item, but of all items, because sooner or later, this task will have to be picked up again.
Non-Value Added time can best be described as the time you spend on a task for which the customer is not willing to pay. Good examples are bug fixes, quality control steps (e.g. testing) etc. Some of this NVA should be eliminated all together, but some of it can be classified as Business Value Added (BVA) time, meaning it is time the client isn’t willing to pay for, but is necessary to keep the system running at a certain quality level. Examples in software development are the creation of release notes, maintaining the task management system, implementing changes throughout the company to create a better service etc.
Overproduction is a typical waste within a software development environment. In my opinion, it exists on 2 levels. The first level of overproduction is scope creep. Scope creep happens when you set off with a defined set of features when you start developing, but after a while additional features need to be implemented as well, or the features change. This will result in additional work that was not foreseen at the start, which in turn leads to longer Process Lead Time (PLT) and longer delivery cycles.
The second level of overproduction can be made aware of by applying the Pareto principle. If we apply this principle, we can say that 80% of your target audience will only use 20% of the features. You will probably spend a lot of time developing features that will hardly ever be used. Does that mean you should not develop them at all? Definitely not! These bells and whistles are often delighters; things that make clients happy. They may result in having an advantage over the competition, attracting more clients. And since you’re in business to earn some money, attracting more clients is always a good thing.
Every software development department has some kind of process that describes and guides the way tasks, feature requests or bug fixes are handled. Having this documentation readily available and understood by the team is a necessity to keep the workflow moving. However, with all good intentions of breaking the process up into many different defined steps for clarity and strictly defined responsibilities, you run into the risk of overprocessing the entire development process.
A process flow needs to be defined, but too many sub-processes or steps within your process will only make it more complex. Increasing complexity causes confusion and sometimes frustration amongst the people that have to follow the process. Everybody hates the government’s red tape, but on the other hand, with overprocessing you introduce red tape in your own working environment! This adds waste to your process because people spent time on things like figuring out the next step, getting all worked up because a colleague has to handle a certain subtask and he’s not available at the moment etc.
Another thing that happens when you have a very complex process flow is that you will have overlapping tasks or responsibilities. Sometimes 2 people will undertake the same action in different steps of the process. Is that really necessary? Can’t you simply eliminate one of those tasks? Sometimes you can’t, because they add an additional quality control step. But I’m pretty sure that in a software development environment you usually can eliminate one of the 2 tasks, avoiding duplicate work and speeding up your process.
The best way to determine overlapping responsibilities is using a Value Stream Map (VSM). In drawing up this VSM, you draw the different process steps and add the responsibilities of each individual for each of these steps. When doing this, it is imperative that you create this map with the entire team. That is the only way you can be sure you are drawing the actual process and not what you think the process is. Thinking the process is exactly the way you think it is, is one of the most common mistakes in creating the VSM.
Defects are probably the easiest to recognize as a waste. Defects is a concept that is well known in the software development business and very easy to explain. A defect occurs when the software product, patch or feature request does not perform as it should be. The term "as it should be" is also defined buy the customer. If the customer isn’t happy with the solution you’re offering, you have a defect.
Now, as you can deduct from the previous sentence, a defect isn’t necessarily a bug. If the client orders or buys a product and it doesn’t fully meet his expectations, you have a defect. If you’re talking about an actual critical bug, then it should definitely be fixed. But not all defects can be fixed. Sometimes you run into limitations that don’t allow you to satisfy the customer 100%. In other cases it’s just not economical or even financially feasible to satisfy all the customer’s needs, and you have to make some tough choices about what to implement and what gets scrapped. The cost of satisfying that specific need would be higher than the return on investment. And of course, if you have multiple clients (which I hope you have), you can’t satisfy each and every one of them to the same extent. And this brings me neatly to the last point I want to convey in this article: Cost Of Poor Quality.
Cost Of Poor Quality is the cost that would disappear when all processes, systems, products and people wee perfect. It is a substantial cost you inherit from all sorts of problems. It is usually compared to a massive iceberg. You only see about 10% of it above water and about the same percentage of COPQ will be really visible at first. But it is the other 90% that lies at the base of your additional cost.
Let me give you a manufacturing example first. Assume we have a defect rate of 10% in our production cycle. That means that to sell 1000 items, you actually have to produce 1100 items. And that means that you’re making 100 items on which you make no money whatsoever. The origin of these defects can lie in a lot of places: defective base components, bad machinery, etc.
Where do we find COPQ in software development? Well, the defects are obvious by now, as we’ve discussed them above. However, the source of those defects can be found in faulty tracking systems, bad management, poorly trained developers, not using the technology as it should be used, lack of documentation, too many system failures, etc.
If you see an issue that can be narrowed down to COPQ, you should continue to dig further and really get to the bottom of it to find the source of the cost. And then you MUST act upon it and not just leave it as a “known issue”. If you really want to improve something, you should grasp every chance you have, because the cost of improving things will earn itself back in virtually no time at all.
Mario Andretti, a former F1 World Champion, once said: “If you feel like you have things under control, you’re just not going fast enough!” This is a motto by which I try to live by and which also applies to lean thinking in my opinion. Because I believe that if you feel comfortable in your situation, that means you have room for improvement.
When it comes to continuous improvement, you should always step outside your comfort zone and find new or better ways to improve what it is you’re doing. After all, it’s called CONTINUOUS for a reason!
About the Author
Steven Peeters is a freelance consultant with an extensive background in both application and web development. As an Adobe Certified Instructor he has also given trainings to various people and institutions in the BeLux region. In 2012 he started his own company Silver Lining with the intention to combine the training aspect with consultancy in project, change and process management. With this goal in mind he started focusing on Lean Six Sigma, becoming a Lean Six Sigma Black Belt and applying these techniques to an IT environment.
Decreasing WIP increases throughput?
I found this statement contradictory: "That means that if you make the process lead time smaller by reducing the waste, you increase the throughput. But this also happens when you reduce the amount of work in progress." . It seems to me that because Little's law is exactly the opposite, if throughput = WIP / Lead time, if you decrease WIP you decrease throughput. Is there something I'm missing?
Of course all this is considering that WIP and Lead time are independents which are not in this particular use of the law.
Re: Decreasing WIP increases throughput?
I think you have the formula a little bit mixed up for my explanation. The Process Lead Time is what determines your throughput, so that is on the other side of the equation. So, the PLT = WIP / PCE, with PCE being the process cycle efficiency. The higher your efficiency of handling one item or task, the lower the lead time will be and therefore the higher your throughput will be.
I hope this makes it more clear.
InfoQ Sep 01, 2015