How Airbnb Uses Net Promoter Score to Predict Guest Rebooking
Net Promoter Score (NPS) is a customer loyalty metric used to determine the likelihood that a customer will return to a company’s website or use their service again. Introduced by Fred Reicheld in 2003, NPS is calculated based on consumers' responses to a question called "likelihood to recommend" or LTR (how likely are they to recommend the company product, service, event to a friend). The NPS is based on the feedback given on a scale of zero to ten.
Consumers who respond with a 9 or 10 are called "promoters"; these are the customers who put their reputation on the line to their friends to promote the company. People who respond with a score of zero to six are called "detractors"; they are the unhappy customers who are likely to tell friends and colleagues the company product or service isn't worth bothering with. Those who give a score 7 or 8 are considered to be "passives"; they liked the company product or service, but they're not going to promote it among their friends.
NPS is calculated by subtracting the percentage of detractors from the percentage of promoters as shown below:
NPS = (% of Promoters) - (% of Detractors)
*Promoters = score of 9s or 10s; Detractors = score of zero through 6
NPS value ranges from -100 (all responses are detractors) to +100 (all responses are promoters).
Airbnb, the company behind the community marketplace for people to list, discover, and book unique accommodations around the world, uses NPS extensively in measuring the customer loyalty. They believe this is a more effective metric to determine the likelihood that a customer will return to book again or recommend the company to their friends.
Lisa Qian from Airbnb engineering team recently wrote about how they use data to understand the quality of the trip experience. They found higher NPS scores generally correspond to more referrals and rebookings.
The team also uses other review categories like Accuracy, Cleanliness, Checkin, Communication, Location and Value to predict the customer rebookings. By comparing a series of nested logistic regression models, they are able to assess the power of review ratings to predict whether or not a guest will take another trip on Airbnb in the 12 months after trip end.
It’s also important to track the response rates when measuring NPS scores.
Here are some interesting statistics of the prediction of customer rebookings. Using only a guest’s LTR at the end of trip, Airbnb team can accurately predict if their guests will rebook again in the next 12 months 56% of the time. By adding the basic information about the guest, host and trip, the predictive accuracy is improved to 63.5%. Then adding the review categories (not including LTR), they add an additional 0.1% improvement.
NPS was originally championed by our CEO and co-founder Floyd Marinescu and was quickly adopted by the QCon Brazil team. Since then we’ve gradually adopted it across our products.
For QCon we ask an NPS question as part of the survey we send out to all attendees after a conference, and, more recently, we’ve also asked the question of InfoQ readers for the first time using the third party Qeryz tool. We were pretty pleased with the score we got - 42%. For reference our highest rated English Qcon was 53%. We consider the numbers we have to be really good but we do have a top initiative in the company at the moment looking to improve them.
In both cases it’s one of a number of measures that we look at but it’s certainly useful. If we see our NPS score dropping it acts as a useful warning sign that there is something we need to investigate and potentially try to address. It’s also great to have something that allows us to compare perception across our different products and geographies.
There is some criticism of NPS that it doesn't add anything compared to other loyalty-related questions. There was no scientific evidence that the "likelihood to recommend" question is a better predictor of business growth compared to other customer-loyalty questions. Other concerns are that NPS uses a scale of low predictive validity, it's less accurate than composite index of questions, and it fails to predict loyalty behaviors.