BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles Visualizing the Big Picture of your Agile Project

Visualizing the Big Picture of your Agile Project

Bookmarks

Agile is all about the “whole team” experience. We plan together, code together, test together, and retrospect together so that everyone in the team is always on the same page. However, once your project grows bigger, teams start to get lost in big pile of user stories and it gets harder and harder for everyone to see the same big picture. This article discusses various ways to visualize this big picture. Not for just the product owner or manager but for everyone.

In the ideal scenario, an Agile team is supposed to be crystal clear about the plan for the current iteration and only have a rough plan for the rest of the release. What actually happens many times is the team rushes to get the current iteration done with little idea of what is coming next. They simply lose sight of the big picture. Often, this picture is kept in a head of a single role like team lead, business analyst, project manager, product owner or even a ScrumMaster if he or she does not really push for self-organizing. This is probably fine in some contexts, like a small short-length project, but in many cases this will lead to a bad end as the team might have no idea that they are getting off the track since all that work makes it feel like they are on track.

Business Value

Most of the time, we build something based on some grand idea - which may come from your company founder, your department head, your group of customers or your community. In the world that we live in, this idea is often not static; it changes all the time. Having the ability to see the big picture of how exactly the project has progressed provides us with more insights to help us pivot to the right direction.

For example: your big boss had an insight that Login by LinkedIn would be a killer feature to penetrate your untapped professional market but the communication got distorted and the priority was twisted due to various factors once it came down to the product owner. The Linked API was not ready. You had 5 other production issues, and lots of other valid reasons but eventually you were halfway to the launch date and LinkedIn integration had not even started. It would be much better for your boss and for your team to know this discrepancy right from the beginning instead of late in the game.

Sometimes teams are stuck on technical issues on not-so-important features , investing way too much for their return of business value. Knowing that the team has spent more than half of the effort trying to solve integration with FourSquare for the last 3 iterations would allow the product owner to say “forget about it” and move on.

So how do we make this information visible?

Burndown Chart

The release burndown chart is a classic progress visualization tool in Agile teams and it is very effective to portray team progress in term of speed and throughput. It can nicely show the summary based on story points and status of these hundreds stories. It has its own unique beauty but it may not be enough. What if you arrive on time but you find out that it is the wrong destination! The burndown chart can tell WHEN it will be done but not WHAT has been built. The fictional burndown chart below from Mike Cohn’s book Agile Estimating & Planning book shows that some scope is added at the end of iteration two and then everything is projected to be back on track.

How do we visualize this big picture? Here are a few techniques.

Epic

An Epic is basically nothing but a big user story. The term has been widely adopted through Mike Cohn and his book Agile Estimating and Planning. However, the term Epic along with similar terms like Theme and Feature are used differently among different Agile teams but they are all techniques to group user stories in some fashion. Mike Cohn mentioned in a comment in his blog that the term Epic was originally explained to him by Kent Beck. Here are some definition from the blog.

There's no magic threshold at which we call a particular story an epic. It just means “big user story”.

“theme” is a collection of user stories. We could put a rubber band around that group of stories I wrote about monthly reporting and we'd call that a “theme.”

This also means that Epic and Theme really have nothing to do with each other. The picture below might give you a better idea.

(Click on the image to enlarge it)

Mike made an interesting comment that “Calling a story an epic can sometimes convey additional meaning” e.g the story is too big to estimate and we need to break it down to smaller stories.

Lean folks also introduced other terms like MMF (Minimal Marketable Feature or Minimal Marketable Feature set) which is a different way of defining requirements. MMF is usually bigger than story and many have equated it to Epic but it has more concrete definition than just a big story. MMF is a smallest set of features that a customer will buy if you release it. If it is not marketable, it is probably too small and it should not be split out of its bigger story. One or more MMFs released together is Minimal Viable Product (MVP). The term has recently been made very popular, due to the rise of the Lean Startup movement.

So we have Epic, Theme and MMF if that is counted. Now what? How do we use them to help us get a better big picture?

Story Mapping

The Story Mapping pattern, which has been made popular by Jeff Patton, is one way of visualizing a large backlog of stories. Using Mike Cohn’s Epic term, Story Mapping is nothing but a map of Epics and all of their child Stories on a big wall. The backbone contains the list of Epics in an order which  you think the story should be told while the children stories are listed down, sorted by priority,  under the backbone like the walking skeleton. The picture below is from The new user story backlog is a map, written in 2008, by Jeff Patton.

Jeff described the usage of Story Mapping as follows.

When the project is running, it becomes our sprint or iteration planning board. We identify or mark off stories to build in the next iteration directly on the map. During the iteration we'll place just the stories we're working on into a task wall to managing their development - but the story map lives on the planning wall reminding us what the big picture is, and how far we've come.

Story Mapping is certainly a good approach to organize your large backlog to better tell a story than simply a flat backlog. Grouping stories under the same backbone item (like Epic) helps us communicate at a higher level than that of at the detailed story. It can also be very helpful when picking MVP pieces since you probably need a (top) bit of each walking skeleton pile to compose an MVP.

However, sometime you just need a single picture which portrays the overview of the project. You showed your boss the burndown chart already but it only tells WHEN not WHAT. You wished your boss wanted to spend the time walking along your big Story Mapping wall but it was not the case. What do we do? Any other options?

Parking Lot Diagram

An interesting big picture visualization technique is the “Relative-Size” Parking Lot Diagram mentioned in Parking Lot Diagrams Revisited – Using Area to Show Effort by Mike Griffiths written in 2009.

(Click on the image to enlarge it)

The traffic light color-coding here represents the urgency e.g. red means it has already missed the schedule while green means it meets the deadline. Most of the items are still in yellow since they are still progressing toward the deadline. The relative size is the improved version of the original same-size Parking Lot diagram. The bigger the box the bigger the estimate (in story points) it has.  This diagram is quite easy to understand but it is very tedious to manually produce in PowerPoint, as the author pointed out. The blog also mentions a few other ideas including a simple scaled bar chart which is easier to produce with Excel or coding.

(Click on the image to enlarge it)

TreeMap

Another visualization which I find makes a lot of sense to capture large and complex data set is Visualizing a Large Product Backlog With a Treemap (aka Heatmap) by Mike Cohn written in 2008. Despite being written a few years ago, Mike’s recent comment still confirms his believes that it is still the best way to visualize a large project.

“Yes, I do still think this is the best way to visualize a large product backlog. Treemaps have stood the test of time for things like visualizing the stock market so I don’t expect them to be replaced as a good technique for us any time soon [Mike Cohn, May 11, 2012].”

This treemap below is derived from current backlog of beta release of Eidos, the agile project collaboration tool that my company is working on,  drawn by Google Chart Tools API. The darker green the block,  the further the epic has progressed and their areas are relative by Epic size (sum of story point of all its stories). This treemap tells us that Eidos is almost ready for beta since many big Epics are complete.

(Click on the image to enlarge it)

These pictures give us a good snapshot of today but it does not tell us anything about the past or the future since there are only two dimensions of the data here e.g. Epic size and status. There is no dimension of time.

Dartboard

The dartboard-like diagram bellows, mentioned in Epic Visualization for a Product by Nicholas Muldoon, visualizes “planning” of each Epic represented by each section. Each square note is a story. The inner circle is the current release and the next 4 outer circles are the next 4 releases. The notes outside the circle are the unplanned stories. The traffic light color-coding represents readiness of the stories ranging from red, to yellow and eventually green.

Though this diagram shows the dimension of time, it does not really show the relative effort among stories. Perhaps, it can easily be enhanced by making the size of each note reflects its relative size.

Stacked Area Chart

For Eidos, we are researching on how to effectively visualize the big picture. One additional idea we have is the Stacked Area Chart below. This chart does not only show relative size and status but also their progress through time. For example, you can see that we have worked on the Storyboard and StoryCard epics from the beginning but we just started putting in IterPlan and Attachment in iteration 6.

(Click on the image to enlarge it)

Enhanced Burndown Chart with Epic Bars

Another idea is to show the Epic progress under the burndown chart as shown in the original Eidos screenshot below. Each color box in the bar charts underneath the burndown chart here represents sum of story points of each epic. In other chart, this enhanced burndown can also show “what are left” to be implemented for each epic.

(Click on the image to enlarge it)

Summary

All these visualizations are trying to solve the same problem e.g. summarize the large and complex set of data in different dimensions including time, size and requirement grouping. The more complex the data and the visualization is, the more automation you need to put this big picture together.

Let’s look at all the visualization we discussed here again. Key differentiations are the dimension which the visualization shows. Obviously, if it shows just the snapshot of time (like Treemap), it will not show the data in time series (like Burndown).

Visualization

Time

Scope & Status Breakdown

Pros

Cons

Programmatically Draw

(Release) Burndown

Time Series

Release Level

Intuitive,
easy to plot by hand

No breakdown data

Hard

Story Mapping

Snapshot

Story Level

Intuitive,
Easy to do by hand

Can be hard to maintain with large backlog

Very Hard (mostly done by post-it on the wall)

Parking Lot

Snapshot

Epic Level

Intuitive

Time consuming to produce

Very Hard

Scaled Bar Chart

Snapshot

Epic Level

Intuitive

 

Time consuming to produce

Easy

Treemap

Snapshot

Epic Level

Very informative to show overall scope and status

Not intuitive

Moderate

Dartboard

Snapshot

Epic Level

Take up small space

Not practical with large backlog

Hard

Stacked Area Chart

Time Series

Epic Level

Capture both time series and breakdown

Not intuitive and somewhat cluttered

Moderate

Burndown + Epic Bar

Time Series

Epic Level

Intuitive

A bit cluttered

Hard

What do you think about these visualizations? How do you visualize the big picture of your project? We would love to hear your thoughts.

About the Author

Kulawat Wongsaroj is an Agile coach goes entrepreneur trying to solve the world problem of inhumane Agile project collaboration tools. In the quest for the best agile tool, none was uncovered so he found Proteus Agility in Thailand with the mission to build Eidos, the humane software for human.  Kulawat is the lead translator of Agile Manifesto Translation project for Thai language and also the founder of Agile66.com, the largest Agile community in Thailand. He regularly preaches about Agile in Thailand tech gathering and conducts Agile training for Thai software companies.

Rate this Article

Adoption
Style

BT