BT

Better Estimations Using Techniques from Psychology

| by Ben Linders Follow 25 Followers on Sep 28, 2016. Estimated reading time: 7 minutes |

Bias, priming, and salience are the main psychological factors that influence our ability to estimate. Knowing what happens psychologically when we estimate, and using techniques from psychology, help us to deal with those factors so that we can improve our estimations, argued Joseph Pelrine, social complexity scientist and PhD researcher in psychology.

At the Agile and Software Architecture Symposium 2016 Pelrine gave the opening keynote in which he talked about the psychological aspects of estimating. InfoQ is covering this conference with Q&As, summaries, and articles.

The language we use primes us when doing estimations, Pelrine stated. The words used when asking for an estimate can result in different figures. He referred to the "reconstruction of automobile destruction" experiment that was done by Loftus and Palmer (1974) where using different verbs like "smashed" or "hit" when asking teams about the speed of the cars led to different estimates.

The way most people calculating velocity by using a moving window average is wrong, said Pelrine. You have to question if all data points are equal in importance when for instance the team composition has changed or if the team has improved their way of working. Solutions that would give better velocity estimates are to reduce the number of iterations for calculating the average or use an exponentially weighted formula.

InfoQ interviewed Pelrine about the cognitive processes that happen in the brains of people when they are doing estimations, why people keep using practices for which there is no evidence that they work, psychological factors which influence people's ability to work, techniques from psychology for doing estimation, why people don’t like to do estimations, and approaches for estimating work.

InfoQ: Which cognitive processes happen in the brain of people doing estimations?

Joseph Pelrine: A full answer would require delving into neuroscience, i.e. the "hardware" of the brain, which is not my field of speciality.

Cognitive processes are the "software" that run on the "hardware" of our brain. They can be roughly divided into two types: the subconscious operating system, and the conscious application layer, although the distinction is often flowing and the interaction seamless.

Estimation is a creative decision-making process that takes place in the subconscious based on a first-fit pattern-matching process which evolved over millennia (Klein, 1999). External inputs go through the conscious application layer and are passed over to the pattern-matching engine, which compares them to our database of experience, makes a decision, and passes that back up to be rationalised and expressed. This process is heavily influenced by three factors: cognitive bias, priming, in which exposure to a stimulus moderates a response, and salience, which is how we consider something to be relevant or important.

InfoQ: Sometimes people continue using certain practices with no evidence of working, or even evidence that they don’t work. What is it that makes people keep on using such practices?

Pelrine: There are many possible reasons. Force of habit. Fear of the unknown. The time required to become accustomed to new ways of working. Just try to get a developer to change his code formatting style, let alone his IDE! "Sharpen your axe!" said the traveller to the farmer slowly chopping wood in the forest. "I don’t have time," the farmer replied, "I need to cut wood."

Humans develop rituals as a way to bring order and structure into their lives. Rituals give a feeling of comfort, a feeling of familiarity, and help to reduce cognitive load, thus freeing up cognitive resources to deal with more urgent challenges.

InfoQ: Which psychological factors influence people's ability to estimate?

Pelrine: Here too, there are many factors. First off the whole realm of memory, which touches on both the neurological and psychological sides of our cognitive processes. Then the three factors mentioned before: bias, priming, and salience.

For me, though, one of the most important factors is the Kahneman and Tversky’s Planning Fallacy (1973), a theory so groundbreaking that it won Daniel Kahneman the Nobel Prize in Economics.

"The planning fallacy is a consequence of the tendency to neglect distributional data, and to adopt what may be termed an ’internal approach’ to prediction, where one focuses on the constituents of the specific problem rather than on the distribution of outcomes in similar cases. The internal approach to the evaluation of plans is likely to produce underestimation." (Kahneman and Tversky, 1977)

InfoQ: How can you deal with those factors?

Pelrine: A first step is simply creating and raising awareness of these cognitive processes, so that they don’t happen purely subconsciously. Be aware and accept that the brain is trying to act in your best interest! A next step would be to use new technical and psychological methods to help increase the quality of the estimation process.

InfoQ: Which techniques from psychology do you recommend for doing estimation?

Pelrine: Most of us are already using one psychologically based technique every day. Known as unpacking (Kruger and Evans, 2004), we know it as task breakdown. Unpacking help raise the quality of estimates through the analysis of all steps necessary to achieve a goal, which helps ensure that nothing gets overseen or let out.

Some other simple techniques are:

  • Avoid distraction while estimating (Day et al., 2009)
  • Consider the time of day – estimation sessions in the morning tend to be of higher quality (Blatter and Cajochen, 2007)
  • Keep estimation sessions short – no longer than 90 minutes. (Broughton, 1975)

A number of techniques which were recommended in Danziger et al.’s (2011) study on ego depletion and decision fatigue are:

  • Take frequent short rests (Tyler and Burns, 2008)
  • Keep a positive attitude and relax during the breaks (Tice et al., 2007)
  • Consume glucose to counter-act the effects of ego depletion (Gailliot et al., 2007)

InfoQ: Many people don’t like to do estimation. Can you explain why?

Pelrine: Estimation is commonly viewed as a chore that must be done, and many question the value of it. Estimates are viewed as commitments to which developers are held by management. The tools and techniques many developers use are often lacking. And in the end you have to accept the phenomena of retrospective coherence, the fact that in a complex system you can’t know things in advance.

InfoQ: Which approaches do you recommend for estimating work?

Pelrine: A very good method is reference class forecasting (Flyvbjerg, 2008), which was developed as a way to apply Kahneman and Tversky’s research to software estimation.

For task-level estimation I’ve developed chronobiological method, nicknamed the "Quattro Stagioni" method, which is based on Kleitman’s research into the Basic Rest Activity Cycle, or BRAC (Kleitman, 1957). Simply explained, the unit of estimation is not an artificial unit such as a story point, but the natural cyclic period of 90 minutes. Essentially, you break the day into 4 quarters of roughly 90 minutes each – start of day to coffee break, after coffee break to lunch, after lunch to coffee break, and after coffee break to end of day – and estimate in these units.

InfoQ: Do you have references of theories or studies that dive more into cognitive and social psychological aspects?

Pelrine: In addition to the studies mentioned above, I highly recommend Daniel Kahneman’s "Thinking fast and slow" (2011) and Dan Ariely’s "Predictably Irrational" (2009), both of which are more easily accessible to the layperson than most psychological papers are.

References:

Ariely, D. (2009) Predictably irrational. Harper Collins.

Blatter, K. & Cajochen, C. (2007) Circadian rhythms in cognitive performance: Methodological constraints, protocols, theoretical underpinnings. Physiology & Behavior, 90, 196-208.

Broughton, R. (1975) Biorhythmic variations in consciousness and psychological functions. Canadian Psychological Review/Psychologie canadienne, 16, 217.

Danziger, S., Levav, J. & Avnaim-Pesso, L. (2011) Extraneous factors in judicial decisions. Proc Natl Acad Sci U S A, 108, 6889-6892.

Day, R.-F., Lin, C.-H., Huang, W.-H. & Chuang, S.-H. (2009) Effects of music tempo and task difficulty on multi-attribute decision-making: An eye-tracking approach. Computers in Human Behavior, 25, 130-143.

Flyvbjerg, B. (2008) Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice. European Planning Studies, 16, 3-21.

Gailliot, M.T. et al. (2007) Self-control relies on glucose as a limited energy source: willpower is more than a metaphor. Journal of personality and social psychology, 92, 325-336.

Kahneman, D. (2011) Thinking, Fast and Slow. Penguin.

Kahneman, D. & Tversky, A. (1973) On the psychology of prediction. Psychological Review, 80, 237-251.

Kahneman, D. & Tversky, A. (1977) Intuitive prediction: Biases and corrective procedures.

Klein, G. (1999) Sources of Power: How People Make Decisions. Cambridge: The MIT Press.

Kleitman, N. (1957) Sleep, wakefulness, and consciousness. Psychological Bulletin, 54, 354-359.

Kruger, J. & Evans, M. (2004) If you don’t want to be late, enumerate: Unpacking reduces the planning fallacy. Journal of Experimental Social Psychology, 40, 586-598.

Loftus, E.F. & Palmer, J.C. (1974) Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 13, 585-589.

Tice, D.M., Baumeister, R.F., Shmueli, D. & Muraven, M. (2007) Restoring the self: Positive affect helps improve self-regulation following ego depletion. Journal of Experimental Social Psychology, 43, 379-384.

Tyler, J.M. & Burns, K.C. (2008) After Depletion: The Replenishment of the Self’s Regulatory Resources. Self and Identity, 7, 305-321.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Taking the outside view by William Anderson

There is some good advice in here, but some of the research sited is also suspect. In particular, notions of priming and ego-depletion have both suffered from replication issues in recent years.

That said, the Kahneman and Tversky concept of internal vs. external perspective is solid gold. Joel Spolsky's "Evidence Based Scheduling"[0] technique is an application of this insight to software planning (though I admit that I have not tried it in practice).

This is perhaps why, in the University of New South Wales study referenced in _Peopleware_, it is noted that teams with third-party analysts providing the estimates are more productive (because the third-party analysts are better able to take the outside view). Of course, the study found that teams with no estimates were the most productive, but that's a different discussion entirely[1].

[0]www.joelonsoftware.com/items/2007/10/26.html
[1]gigamonkeys.wordpress.com/2007/04/26/estimation...

Re: Taking the outside view by Joseph Pelrine

An interesting comment.

You state that “some of the research sited (sic) is also suspect” presumably as being a fact, but you provide nothing to back up that claim. A professional would have stated exactly which papers he was referring to, and where the problems were. An informed amateur might have at least noted the importance of replication in advancing science, not only in psychology, but also in other fields of research. They might have looked at both sides of the debate (Gilbert et al., 2016; Open Science Collaboration, 2015), they might have tried to understand the collateral issues involved (researchers’ academic reputation, tenure, grants, funding, etc.), and they might have gone to the Open Science Collaboration website (osf.io/ezcuj/wiki/home/), looked in the database, and would have seen that none of the papers I referenced were noted as having replication issues.

Instead, these comments read like the opinion of someone who bases his knowledge on the popular press, dressed up to seem authoritative by throwing in a few buzzwords.

Is this a red herring or a dead salmon?

In addition, although I highly respect Spolsky, DeMarco, and Lister, none of them are psychologists, and the focus of this interview was on psychology. Also, regarding the University of New South Wales study (Jeffery and Lawrence, 1985) you referred to in Peopleware (DeMarco and Lister, 2013), it’s always better to cite primary sources. The study itself is also quite interesting reading, and deals with many issues besides estimating.

References

DeMarco, T. & Lister, T. (2013) Peopleware: Productive Projects and Teams. Addison-Wesley Professional.
Gilbert, D., King, G., Pettigrew, S. & Wilson, T. (2016) Comment on “Estimating the reproducibility of psychological science”. Science, 351, 1037a-1037b.
Jeffery, D.R. & Lawrence, M.J. (1985) Managing programming productivity. Journal of Systems and Software, 5, 49-58.
Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science, 349, 943-952.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

2 Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT