Knowing if You Are Building the Right Product
Developing and delivering products which customers don’t want and for which there is no market can be costly. Agile can help you to efficiently develop products, but you need to know what to build. How can you find out which products your customers need?
How sure are you that you are producing the right thing? How do you decide what is your next best investment, and how do you validate your choice? (…) What makes you sure, and what makes you unsure, are areas of thinking and learning that confront teams when they’ve worked hard to smooth out delivery.
The Definition of Done help teams to check if the software is ready to be delivered. But it can give a false sense of certainty as David explains:
Unfortunately, products are only done when they are in use. Watching users in the wild often teaches teams that what they were certain about. “I am sure people will need to …” is not what people need. It may be that one person’s arrogance, or fear of “not being a good product owner,” is the issue. It could also be the simply fact that the product ideas were right and the market changed the game. When this happens, shedding arrogance and embracing evidence is your best tool for building less of the wrong thing (which allows you to learn fast and spend less).
In the blog post the problem with building products in an agile world Paul Young explores several problem areas of agile from a product management standpoint. Agile values feedback from customers, but the feedback that you get doesn’t always tell you if you are building the products that the market needs:
In my experience, what actually happens is that most teams recruit four to five customers who are willing to provide that level of constant feedback and then roll with them – for the rest of the release or even multiple releases.
Going back to the same customers for feedback creates a two-fold problem. First, do those four to five customers really represent your overall market segment? Or, are they the “noisy 20%” of your market segment? In my experience, the types of customers who are willing to sign up for regular meetings to provide feedback every other week don’t typically represent average users; they represent power users and experts.
Second, customers aren’t dumb. Soon, those four to five customers figure out that they have a disproportionate amount of influence on your direction, and realize that their feedback can turn you into essentially a custom development shop for their particular needs – again, that’s not market driven, that’s customer driven.
Another problem Paul described is that implementing the product manager role can prevent an organization from being market driven:
Unfortunately, the Product Owner role is extremely noisy and tactical and requires a lot of day-to-day interaction with Development in daily standups, sprint planning sessions, and retrospectives. While there is nothing wrong with making the Agile team run efficiently, I have seen teams get so obsessed with “making Agile work” that they start sacrificing what Product Management really should be doing – which is spending time with the market.
Paul warns us that being too focused on the agile practices and ceremonies can result in building the wrong products:
Agile’s big promise is that it will help us run faster in Development (or whatever), but it doesn’t say anything about if we are building the right thing to begin with. That is the role of Product Management to find out. I see Agile teams often getting so obsessed with the machinations of Agile that they forget that you can be the most efficient company in the world, but if you’re building something that no one wants, it’s pointless.
In the blog post Scrum's flawed notion of product owner on Dr.Dobb’s, Allen Holub discusses the difference between having on-site customers or working with product owners:
One of the many things about Scrum that makes me uncomfortable is the notion of a Product Owner (PO), or more precisely, the lack of an on-site customer. The PO job typically mixes the role of customer surrogate with that of marketing-department or product-development representative. The PO "owns" the product in the sense that he or she makes decisions about content, the value of a story to the business, and so on.
Given that XP, from which most of Scrum derives, calls for an actual flesh-and-blood customer on the team, I both wonder where the PO notion comes from and am dismayed by the damage it can do.
Allen advocates that having product owners in stead of on-site customers can result in not knowing sufficiently which products should be build:
Since the PO is not an actual end user of your product, he or she probably won't know the answers to your design questions. You're now building on guesswork rather than facts. When the PO guesses, you implement the wrong thing. When the PO spends a day or more getting the answer, you lose a day's work (more if you had proceeded on a wrong guess and then had to undo whatever you did yesterday). You don't have time not to have a customer on the team. When the PO is a surrogate customer, the odds of building the wrong product are much higher, and that's a bring-the-company-down failure.
Joel Amoussou wrote a blog post on building business sustainability in which he explores how Lean Startup can complement agile to increase the knowledge of the products that need to be build. Agile can help teams to successfully deliver software products but a purely ceremonial approach may not yield best results as Joel explains:
(…) having the best software developers in town and building a well-designed and well-tested software on time and under budget will not necessarily translate into market success and business growth and sustainability. So what is the missing piece? How do we ensure that Agile is delivering a product that users are willing to buy? How do we know that the software itself and the many features that we work hard to implement every sprint are actually going to be needed, paid for, and used by our customers?
A reason that agile projects fail is when the wrong product is developed:
The issue is that most of the time, neither the Product Owner nor the business users are the people who are actually going to sign a check or use their credit card to buy the product. This is the case when the product will be marketed and sold to the market at large. Implementing the wrong design and building the wrong product can be very costly to the enterprise. So the result is that the design and the features that are being implemented by the development team are just assumptions and hypotheses (by so called experienced experts and product visionaries) that have never been validated. Not surprisingly, many Agile projects have become big failed experiments.
Lean Startup principles can be used to increase the knowledge needed to develop products that customers need:
The Lean Startup recipe is to create Minimum Viable Products (MVPs) that are tested early and often with future customers of the product. An MVP can be created through rapid prototyping or by incrementally delivering new product features through Continuous Delivery, while leveraging cloud-based capabilities such as a Platform as a Service (PaaS) to remain lean. Testing MVPs requires the team (including software developers) to get out of their cubicles or workstations and meet with customers face-to-face whenever possible to obtain direct feedback and validation. MVPs can also be tested through analytics, A/B testing, or end user usability testing. Actionable metrics like the System Usability Scale (SUS) should be collected during these fail-safe experiments and subsequently analyzed.