The Current Direction of Agile

In the last couple of years, Agile adoption has increased quite a bit, letting a number of people experience and challenge different elements of the Agile processes. This increased exposure and usage has made some try out different solutions to different challenges than what was originally proposed by the people that came up with the Agile processes. This is a core part of Agile, to change the process over time and make it more effective. This article will focus on some of the recent trends within the Agile community by briefly describing some alternatives to today’s well known Agile processes. Particularly focusing on estimation, forecasting deliverables and the increased impact Lean manufacturing has had on the Agile community.

Lean

I will not cover Lean specifically, but I will briefly cover some of the Lean principles, practices and tools which are relevant to the other processes and tools covered in this article. Let’s first look at a brief description of Lean from Wikipedia:

Lean software development is a translation of lean manufacturing principles and practices to the software development domain. Adapted from the Toyota Production System, a pro-lean subculture is emerging from within the Agile community.

Related Sponsored Content

Related Sponsor

Uncover emerging trends and practices from domain experts. Level-up your skills at QCon New York (June 13-15, 2023)!

Lean in software development originated in Mary and Tom Poppendieck’s book, Lean Software Development, where they cover 7 principles and 22 tools of Lean Software Development.

Below is a short list of terms often used in Lean that you should be familiar with reading this article:

• Waste
• Just-in-time (JIT)
• Work In Process (WIP)
• Pull based systems

Waste

Waste (a.k.a. muda), is about eliminating artifacts from your process that does not add value to your customer. Mary Poppendieck defines waste as:

Anything that depletes resources of time, effort, space, or money without adding customer value.

And value as:

Seen through the eyes of those who pay for, use, support, or derive value from our systems.

In Mary’s article, Principles of Lean Thinking (PDF) and also in the Poppendieck book mentioned earlier, she identifies seven wastes of software development. The wording of these wastes has changed a bit since they first were published and below I’ve listed the ones I saw her use in a presentation recently:

1. Extra features
2. Handovers
3. Work in process
4. Failure demand
5. Task switching
6. Delays
7. Defects

(Limit) Work in process (WIP)

Limiting WIP is about having feature requests that have been initialized (specification, requirements gathering, etc.) to flow through the system in a continuous way. In other words, when your organization first starts to work on a feature request, limit how long it stays in process before going to release.

Just-in-time (JIT)

JIT is about reducing in-process inventory and avoid/discover bottlenecks in the process. JIT is often used as a mean to reduce WIP (described above). A typical example of inventory in software development is requirements going from specification to development or code going from development to test. JIT sets focus on improving return on investment (ROI), quality and efficiency. One of the processes/tools supporting JIT is Kanban, which I will talk about later in this article.

Pull based systems

The number one identified waste mentioned above is extra features. To pull instead of push is about responding to customer needs, instead of giving the customer what you think he will need (extra features). As with JIT, Kanban is one example of a tool supporting a pull based system.

Iteration Based Processes

Scrum and Extreme Programming (XP) are two well known Agile processes that utilize iterations or sprints as a central part of their planning and deliverable technique. In relation to estimation, you typically find that you estimate User Stories (XP) or Backlog Items (Scrum). In this article I will use User Story to describe both User Story and Backlog Item. User Stories are typically estimated in either Ideal Days or Story Points. Tasks are typically described in ideal hours. Mike Cohn, in his book: Agile Estimating and Planning, has covered the different methods and techniques for estimation and forecasting thoroughly.

The main idea is to have fixed length iterations of 1-2 weeks. Figure 1 shows three iterations from a backlog with stories of varying size. It also shows that the team has a velocity of 20 points, meaning their able to deliver 20 points worth of stories every iteration.

Knowing the team’s velocity is important to be able to forecast when a user story will be finished or predict which stories will be part of a release.

Equal Sized User Stories

When an Agile team first starts out they often start with Scrum or XP. This is the approach covered in most Agile books. A typical scenario for a fresh Agile team might go something like this:

The first planning meetings takes quite some time, with a lot of discussion and thinking to come up with the best estimates possible. First for each story, then for the tasks, and at the end decide on how much to commit to for the iteration. After a while (several iterations) when the team gets more experienced, they know almost by instinct how much they can commit to for a single iteration, and estimation of stories and tasks goes much quicker.

Some teams have taken this to another level and skipped the whole estimation process. Rather than spending time on coming up with estimates, they split their user stories into equal sized stories (see figure 2). The main difference is that you don’t estimate each and every story, but you have each story fit a fixed estimate or size.

By having these fixed sized stories, the team can commit to a certain number of stories for their iterations. Their velocity then becomes the amount of stories they are able to commit to, instead of the amount of Story Points or Ideal Days. This simplifies the planning process quite a lot.

Think about how you would find out how many stories (or which stories) a team will get done in a three months time, with a prioritized backlog. Using the team’s velocity (number of stories), multiply it with the number of iterations for the three month period and you have your forecast.

Fixed sized user stories eliminates waste by not spending time on doing estimates and allow the team to focus on the details of what’s important; implementing the stories.

Some argue that you still estimate, since you have to make all stories equal size. How do you do that without estimating? I find this to be about two things:

1. Getting each and every story small enough.
2. The average sized story is what is important.

For the first point it’s important to have as small stories as possible, something that adds value to the business and are deliverable. A two day story is much easier to spot and be correct on its size, than a bigger story that you might think take 8-10 days, but in reality takes 15.

For the second point it does not matter if one story or another is bigger or smaller as long as it’s not extreme deviations. Typically you find an exception from time to time, but all in all your velocity does not fluctuate very much.

Naked Planning

Naked Planning, coined by Arlo Belshee, takes a different approach where not only estimation is removed from the process but also iterations. In relation to estimation Arlo asked himself the question:

Are estimates necessary to decide the order of prioritization and make a guess at how long it’s going to take something to come live?

After talking to the business he found it was not and found another way of accomplishing the same thing.

Inspired by Lean practices, Arlo used a queue of Minimum Marketable Features (MMF’s). Using this queue the development team grabs the top most MMF and start working on it. When finished they grab the next one from the top and so on. Like Lean this is a pulled based system. The queue is managed by the business, always having the most important and valuable MMF at the top. The queue is kept to about seven items, which is about how many a person can keep in his head at once, and also a manageable amount of items to prioritize amongst.

To estimate when an MMF can be delivered they use historical data. When an MMF gets pulled from the queue it’s marked with the date, this is also done when the MMF is finished. Every three to five weeks they calculate the average of the duration, giving the business a number to work with in planning. If the size of the MMF’s put into the queue gets bigger or smaller, the duration estimate adjust itself after a short period of time.

Kanban

From Wikipedia:

Kanban (in kanji 看板 also in katakana カンバン, where kan, 看 / カン, means "visual," and ban, 板 / バン, means "card" or "board") is a concept related to lean and just-in-time (JIT) production.

This description can lead you to believe that Kanban is all about the task board most Agile team’s use. That is not the case. Kanban as a tool uses a white board to visualize the flow of “materials”, but it is not to be confused with the board itself.

Kanban is a way of controlling (limiting) WIP and keeping continuous flow. You use the whiteboard to visualize the different stages an item needs to pass through before being deployed. By monitoring this workflow it’s easier to discover bottlenecks that slows down your process. You typically see this by observing where elements queue up. Often a number is put on columns defining how many items are allowed to be “in process” at the same time. This number is specifically for limiting work in process.

A simple “Kanban” we often see on Agile teams have the following columns: ToDo, In Process and Done (see Figure 4).

Kanban takes this to a different level and visualize the workflow used by the organization and not just the development team (see Figure 5).

Typically you can follow a feature on the Kanban board from the idea and all the way through the different phases in the organization to its deployment into the system. Henrik Kniberg has a nice cartoon like description of a small Kanban system that helps visualize this further.

Kanban does not utilize iterations, but often has regular release cycles. Having a release cycle of e.g. two weeks, you would put whatever has reached the “release state” into production. However, there is nothing preventing you from releasing finished content right into production, as long as your process support one-click deployment, which any Agile team should do.

One thing to note about Kanban: there is no perfect Kanban system that everyone should adapt. It will be different for every organization, because they are different by nature.

David Anderson is one person in the Agile community that has used and documented Kanban in software development. I encourage you to check out his writing and presentations on the subject.

Scrumban

Scrumban was coined by Corey Ladas. Based on his extensive Scrum experience he describes in his book, with the same name, how you can start adapting lean thinking to Scrum focusing on Kanban. Scrumban is interesting because it does not just explain what Kanban is, but takes you through the steps of moving from Scrum towards Kanban in a step-by-step manner. Scrumban is neither Scrum nor Kanban, but if you go all the way you end up with doing Kanban on an Agile team.

Conclusion

There are obviously other examples of where Agile is going not covered here, but I find that these tell a story of which direction Agile is going. Many teams have found iterations to be a limiting factor and looked for ways of removing them. Some wanted to scale the Agile process to not just include the development team and those working closely with that team, but to also include the rest of the organization. Others found that the amount of time spent on estimating was considered waste and found other ways of delivering software and still be able to satisfy the questions and demands from the business.

But all in all I think it shows that the software industry is maturing. Software teams see value in Agile processes, but they continuously want to improve it, which hopefully brings us to better and more effective processes as well as better quality software. Software development is still a young craft and we live in interesting times!