BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

Agile 2012 Session: Stop Listening to Your Customers

by Shane Hastie on Aug 14, 2012 |

Brandon Carlson gave a talk at the Agile 2012 conference titled "Stop Listening to Your Customers"  in which he made the case for using deep analytics and application instrumentation to discover customer behavior and use that to help guide product requirements.  He spoke about the impact of wasted effort and unused features and how just listening to customer requests will result in unused, bloated products and how commonly used requirements techniques have built-in biases which lead towards building products that don't met customer expectations.

He started by stating that many of the technical problems ("building the product right") facing software development have been solved, teams today know how to build good software.  The area where we still fall down is in deciding what features to build into our products ("building the product right").  He discussed the Standish Group survey results that indicate that 64% of features in a typical software product are seldom or never used. He maintains that this is because we listen to our customers.

He went on to tell the story of a mapping software product where there were many customer requests for a printed wall map feature - easy to build in the software but which would necisitate the company spending $150000 on a large format printer and a new logistics process to print and ship the printed wall maps.  Instead of building the feature as requested and implementing the new business process they initially designed the featre to send an email with the map image file to Brandon to take to the local copy shop and have printed off site.  Over the next six months the feature was used a total of three times.  They then changed the feature to send the email with the image file back to the customer and give them the capability to print their maps at a local copy shop, saving the organisation thousands of dollars on a feature which was very seldom used.

He maintains that it is actually important to listen to what customers state they want, and even more important to understand what they actually do - combining listening with seeing in the field.

He went on to explain that the most common approaches (interviews, focus groups and surveys) used for listening to customers have bias and problems built in to them:

  • Response Bias - when asked a question most people will respond with what they think you want to hear, rather than what they need.  He used the example of a question such as "would you want this available on the iPad" to which most people will respond with yes, even if they do not own an iPad.
  • Social Norms - people will often hide their true motivation behind polite facts  For example when asked "why did you purchase xyz product over abc product" the reason given may list features which appear to be missing, but the actual reason may be as simple as "I didn't like the sales person".  The list of missing features becomes an enhancement request, whereas it was actually just a convienent way of deflecting a difficult question.
  • Stated & Revealed Preferences.  "Actions speak louder than words".  He discussed the question "do you recycle" to which most people will respond "yes", but observing their behavour it becomes clear that for many people the response is actually "yes, I think recycling is a good idea and I will do so if it is easy and convientent, but not if it requires much extra effort or inconvienence on my behalf".  All interviews, focus groups and surveys provide information on stated preferences - revealed preferences can only be found by observing behavour.

He went on to discuss how to find revealed preferences as that will provide the best information to decide on features and capabilities of products.  Specific ideas he suggested are:

  • Within the current application - use log files and other analytic tools to see what actions users actually take and then explore what they were trying to achieve.
  • Within other internal applications with similair characteristics - again using analytics and logs to understand what users do and then explore why they behave in certain ways
  • From Beta softeare releases - provide new functionality to a small group and monitor their actual usage of features to see if they behave as expected
  • Application instrumentation - buildnew logging and/or analytical capabilities into the product to answer specific questions about use and behavour.  He strongly advices keeping this simple

He summed up his talk with two pieces of advice:

  • Be skeptical - what is asked for is frequently not what is actually needed
  • Use the data - tools and analytics can identify behavours and needs which must be examined further

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Typo by Chaz Allen

I think your second "building the product right" should read "building the right product"

Listening deeply by Dave Nicolette

I wonder if the problems Brandon notes are caused by shallow listening. His suggestion to use analytics to discover what customers actually use and need could be understood as a method of deep listening. We listen not so much to what customers say they want as we do to what their behavior tells us they need. Very pragmatic. I like it.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

2 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT