Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

### Topics

InfoQ Homepage Podcasts Conal Scanlon on Monte Carlo Mapping

# Conal Scanlon on Monte Carlo Mapping

In this podcast Shane Hastie, Lead Editor for Culture & Methods, spoke to Conal Scanlon about his talk at QCon New York on Monte Carlo Simulation for forecasting feature development.

### Key Takeaways

• Knowledge work is inherently variable, and estimates are inevitably incorrect
• Monte Carlo simulation projects likely completion based on past history rather than future guesses
• A small set of real data points is extrapolated to 1000 samples and that is used to produce a probability curve
• A forecast is a point in time situation – as teams get better at delivery their predictability should improve
• Everything in the delivery process should be subject to change as it is continuously improved
• 0:30 Introductions
• 1:00 The product manager role as an interface between engineering and customers
• 1:15 Knowledge work is inherently variable, and estimates are inevitably incorrect
• 1:45 Monte Carlo simulation projects likely completion based on past history rather than future guesses
• 2:05 The five steps of Monte Carlo simulation
• 2:12 Gather past data – Cycle time for a feature
• 2:33 Extrapolating from a small set of real data into 1000 different samples
• 2:57 The statistical technique of bootstrapping
• 3:20 Once the sample set is available use the size of the backlog to aggregate how long individual items will take to complete
• 3:37 Using medians and confidence levels to forecast likely completion times
• 4:00 How this differs from just counting the number of items in the backlog
• 4:15 Building in the variability that is inherent in knowledge work
• 4:45 Developer estimates do not corelate well to reality
• 5:44 Explaining how the small set of real data is extrapolated to 1000 samples and how that produces a smoother probability curve
• 7:25 This doesn’t need a statistician to accomplish the maths – it’s relatively straightforward
• 7:40 Explaining how to accomplish this on your own data
• 8:05 The need for clarity about what is being forecasted – what is your definition of cycle time
• 9:14 Using cumulative probability distribution charts to convey the uncertainty in a forecast to stakeholders
• 9:54 Using the chart to identify dates you are comfortable publishing internally and externally
• 10:52 This approach overcomes the resistance and fear of giving estimates as it is forecasting based on trends using historic data
• 11:08 Using story mapping to identify the relative complexity of features
• 11:42 Describing the story mapping technique and how conveys the complexity of a feature
• 12:48 The need to educate and socialise the approach and that the numbers are probabilities and forecasts not guaranteed delivery dates
• 13:20 Ideas from the book How to Measure Everything
• 13:40 Techniques to convey uncertainty and the fallibility of estimation
• 14:55 Using the forecasts and real data to examine the effectiveness of your development process and the underlying assumptions that are being made
• 15:25 The Product Manager is responsible for the clarity of external communication from the team about the forecasts
• 16:05 A forecast is a point in time situation – as teams get better at delivery their predictability should improve
• 16:35 Everything in the delivery process should be subject to change as it is continuously improved

Mentioned: