BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Podcasts Conal Scanlon on Monte Carlo Mapping

Conal Scanlon on Monte Carlo Mapping

Bookmarks

In this podcast Shane Hastie, Lead Editor for Culture & Methods, spoke to Conal Scanlon about his talk at QCon New York on Monte Carlo Simulation for forecasting feature development.

Key Takeaways

  • Knowledge work is inherently variable, and estimates are inevitably incorrect
  • Monte Carlo simulation projects likely completion based on past history rather than future guesses
  • A small set of real data points is extrapolated to 1000 samples and that is used to produce a probability curve
  • A forecast is a point in time situation – as teams get better at delivery their predictability should improve
  • Everything in the delivery process should be subject to change as it is continuously improved
  • 0:30 Introductions
  • 1:00 The product manager role as an interface between engineering and customers
  • 1:15 Knowledge work is inherently variable, and estimates are inevitably incorrect
  • 1:45 Monte Carlo simulation projects likely completion based on past history rather than future guesses
  • 2:05 The five steps of Monte Carlo simulation
  • 2:12 Gather past data – Cycle time for a feature
  • 2:33 Extrapolating from a small set of real data into 1000 different samples
  • 2:57 The statistical technique of bootstrapping
  • 3:20 Once the sample set is available use the size of the backlog to aggregate how long individual items will take to complete
  • 3:37 Using medians and confidence levels to forecast likely completion times
  • 4:00 How this differs from just counting the number of items in the backlog
  • 4:15 Building in the variability that is inherent in knowledge work
  • 4:45 Developer estimates do not corelate well to reality
  • 5:44 Explaining how the small set of real data is extrapolated to 1000 samples and how that produces a smoother probability curve
  • 7:25 This doesn’t need a statistician to accomplish the maths – it’s relatively straightforward 
  • 7:40 Explaining how to accomplish this on your own data
  • 8:05 The need for clarity about what is being forecasted – what is your definition of cycle time
  • 9:14 Using cumulative probability distribution charts to convey the uncertainty in a forecast to stakeholders
  • 9:54 Using the chart to identify dates you are comfortable publishing internally and externally
  • 10:52 This approach overcomes the resistance and fear of giving estimates as it is forecasting based on trends using historic data
  • 11:08 Using story mapping to identify the relative complexity of features
  • 11:42 Describing the story mapping technique and how conveys the complexity of a feature
  • 12:48 The need to educate and socialise the approach and that the numbers are probabilities and forecasts not guaranteed delivery dates
  • 13:20 Ideas from the book How to Measure Everything
  • 13:40 Techniques to convey uncertainty and the fallibility of estimation
  • 14:55 Using the forecasts and real data to examine the effectiveness of your development process and the underlying assumptions that are being made
  • 15:25 The Product Manager is responsible for the clarity of external communication from the team about the forecasts
  • 16:05 A forecast is a point in time situation – as teams get better at delivery their predictability should improve
  • 16:35 Everything in the delivery process should be subject to change as it is continuously improved

Mentioned:

More about our podcasts

You can keep up-to-date with the podcasts via our RSS Feed, and they are available via SoundCloud, Apple Podcasts, Spotify, Overcast and the Google Podcast. From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.

Previous podcasts

Rate this Article

Adoption
Style

BT