Oct 25, 2010 7:41 AM
Is Net Promoter really the Ultimate Question?
-
Like (1)
An article on Fast Company's blog yesterday asks, "Is Net Promoter Really the Ultimate Question?" For CDW -- one of Forrester's 2010 Voice of the Customer Award winners -- the answer is "no." The article describes how the company moved beyond NPS by adopting Walker Information's loyalty index, which is based on three questions rather than one.*
The value of the index approach is twofold: it correlates more highly to the company's financial performance than does NPS alone, and it more effectively uncovers the drivers of relationship health.
This confirms two broad observations from my VoC work here at Forrester. First, likelihood to recommend is not a great high-level metric for some companies/industries because word of mouth isn't always a key business driver (think of utilities and health plans). Likelihood to recommend might still be a decent barometer of relationship health in those cases, since, conceptually, customers won't want to recommend you if they are unhappy with the relationship. But that doesn't make likelihood to recommend an appropriate "North Star" or "rallying cry" for the company. Second, NPS (customer sat, or any high-level metric) won't drive improvement unless you pursue the analysis and improvement methodology underlying it. In other words, just following a high-level metric won't help you change anything. Net Promoter can help drive improvement when used as a program rather than just as a metric.
It's also interesting to note that CDW points to its 1-to-1 closed-loop activities as the biggest (or most measurable) points of value within its VoC program. These activities include following up with customers who rate the relationship poorly and following up with customers who express interest in new products. The article describes the value of closing the loop for service recovery as follows: "When we first started doing this, the customers were surprised ...just resolving something as simple as a shipping problem results in higher loyalty." The value of closing the loop for relationship enrichment is even more (financially) compelling: "Through this research, CDW identified 12,000 customers interested in specific offerings that were passed onto the sales team. These leads were turned into 200,000 quotes and 108,000 orders placed, amounting to a whopping $230 million in additional revenue." Wow!
What have your experiences been like with NPS? Is it really your ultimate question? Has it helped you make improvements and drive financial results?
*In my original post, I indicated that Fred Reichheld, the creator of Net Promoter, was involved in the creation of Walker Information's loyalty index. This detail was cited in the Fast Company article that inspired the discussion here, but it is not accurate. I apologize for any confusion.
Message was edited by: Andrew McInnes
From my point of view, the ultimate question should be more based on behaviour based facts rather on indications. The fact that Net promoter "indicates advocacy" while with the power of social media the real, behaviour based advocacy can truely be measured. I would love to see extension of NPS, to also include the new possibilities with social media.
The sources used for this are such as Enhancing Net Promoter Score (NPS) with Total Social Customer Value (TSCV) http://www.web-strategist.com/blog/2010/06/20/enhancing-net-promoter-score-nps-with-total-social-customer-value-tscv/ and Online Promoter Score (OPS) http://www.motivequest.com/main.taf?p=6.
Jani Sjölund - twitter.com/zoros1976
Hi Andrew - we definitely see the points you raise as critical questions for companies using MarketTools CustomerSat. Breaking things down a bit:
There are many different viewpoints on this. "Likely to recommend" can be very valuable, but has its limitations as a single metric and does not work equally well in all industry sectors. Joe Camirand, VP of Research and Consulting Services for MarketTools CustomerSat recently posted a blog on the Secure Customer Index, originally developed in 1996 by D. Randall Brandt, PhD. This index combines "Likely to recommend" with "Likely to repurchase" and "Overall satisfaction" metrics. These additional questions provide deeper insight into customer loyalty, allowing businesses to analyze Top Box scores (percent of respondents who choose the highest options in your rating scale) and perform segmentation analysis using the Apostle Model, which correlates overall satisfaction with likelihood to repurchase. Whichever method you use, we always advocate performing analysis beyond the top level score to understand which segments and key drivers provide the greatest opportunity for driving meaningful improvements. Equally important is sticking with a set of key metrics that you can trend over time.
As you highlight, “following a high-level metric won't help you change anything”. I couldn’t agree with you more. Our customers find that by using Enterprise Feedback Management technology, they can gather customer feedback in an objective way that promotes fact based decision making. The right technology can help business leaders quickly identify the best opportunities for investment. From there, however, companies must take action and initiate programs that improve the customer experience.
The CDW example you highlight is inspiring. We’re a huge believer in the value of closing the loop with customers through 1:1 follow-up. Organizing this effectively through a combination of email alerts, case management, and case analytics helps businesses get the maximum return from valuable sales and customer care resources. Often customers will turn out to be more loyal after an issue is handled well than before the issue occurred at all. And correlating 1:1 follow-up actions with resulting new business and customer saves can help you quickly show ROI on your investment in a Voice of the Customer program.
Justin Schuster
VP, Enterprise Products at MarketTools
Thanks, Jani and Justin, for the thoughtful responses.
I agree that it often makes sense to fortify intention-based metrics with behavior-based ones. In terms of likelihood to recommend, in particular, it is intriguing to think about the new possibilities that better social media monitoring can/will enable. Has anyone been successful in using the approach that Jani describes? What about other approaches for combining intention- and behavior-based metrics? As with most things, it's easier said than done.
Justin, I couldn't agree with you more.
I also want to take this opportunity to correct a factual error in my initial post. I had indicated that Fred Reichheld, the creator of Net Promoter, was involved in the creation of Walker Information's loyalty index, which is used by CDW. This detail was cited in the Fast Company article that inspired the discussion here, but it is not accurate. I edited my initial post, and I apologize for any confusion.
Customer behaviors are complex. Companies' differentiators and positioning are varied. Net Promoter offers a simplified (or dumbed down) way to get a metric. However, truly understanding what that metric means for your company and your customers and using it to promote positive change is still complex.
If you're a big company, either way you go, you need smart people and insights beyond just Net Promoter. If you try to oversimplify something complex, you can easily end up with a mess. One thing that can provide decision makers with simplicity and drive positive change is a well designed dashboard for reporting key metrics, insights and recommended actions. I believe better data visualization and use of well designed dashboards will flip this problem on its head.
If you're not too big, I think finding simple metrics that work for your business is a smart thing to do. Maybe Net Promoter is an easy fit for your industry, maybe not.
On a separate note, this prompts a bigger question around experiences... what are the intangible values we wish we could measure around customer experiences?
Brian Regienczuk
I think your point about intangibles is a great one, Brian.
In its broadest sense, the "intangibles" bucket is pretty huge. I could even see satisfaction -- probably the most common customer experience metric -- as an intangible, since it is a feeling rather than a rational calculation, behavior, or intended behavior. However, satisfaction seems pretty tangible because we can (at least try to) measure it just by asking customers whether they feel satisfied. We take the same approach to measure things like trust. The great thing about surveys is that they let us turn feelings into numbers.
But I think you were referring to the smaller group of intangibles -- the factors that customers aren't consciously aware of or can't directly percieve. That's where other methodologies like ethnographic research come in. The measurement problem is that those methodologies don't yield numbers. I often wonder: is that a real problem; or does it just highlight a flaw in companies' thinking, namely an obsession with numbers?
In related news, my colleague Roxie Strohmenger is starting to research things like neuromarketing that can help measure "intangibles."
I'd still love to hear stories from the community about how you are using Net Promoter and whether you've made any modifications to it!
Here's a nice article on how Volusion, an Austin-based maker of e-commerce software, listens to the voice of their customers for generating product ideas on top of using Net Promoter scores.
http://www.inc.com/magazine/20101101/tapping-customers-for-product-ideas.html
Here's another article showing how three companies - First Direct, Marks & Spencer, and Amazon - use customer insights to improve the overall customer experience.
How are you incorporating customer insights into your overall customer experience strategy?
Speaking of intangibles, I just read a short article on this from 1-to-1 Media: http://www.1to1media.com/View.aspx?DocID=32627&From=RssDocument&utm_source=RSSDocument&utm_medium=RSS&utm_campaign=11-03-2010. The majority of the article is an excerpt from Philip Graves' book, Consumer.ology.The basic premise is that traditional market research falls short because consumer behavior is irrational and therefore can't be consciously explained by consumers. The excerpt uses the familiar New Coke debacle as an example of this problem in practice -- people choose to drink Coke for lots of reasons beyond the taste, so market research about taste led Coke in the wrong direction.
I'm curious, does anyone combine survey data such as Net Promoter Scores with more behavior-based analysis to account for intangibles?
Net Promoter is the ultimate question if the only type of behavior you are trying to promote is loyalty or evangelism. If that's the only kind of customer behavior you are interested in, I guess you could be as narrow as a single data point. Generally we hope to engage the customer along the whole buying cycle and there are many different characteristics we'd hope to know to help move them to a fully involved relationship, . http://bit.ly/drFvYT
I'm also a big fan of NPS. It's a key metric in our customer centricity journey. We also ask customers to score their likelihood to repurchase the solution and likelihood to renew their support maintenance agreement. Why?
These additional questions allow us to identify "trapped" customers who rank the likelihood to renew maintenance higher than their likelihood to repurchase the solution. They're not satisfied with the solution; but because of high switching costs (or other barriers), they have to renew the maintenance. We can send in a team to help these customers.
These questions also allow us to calculate the business value of promoters, passives. detractors and a customer centricity initiative focused on improving the customer experience and turning detractors and/or passives into promoters. Being able to put a dollar value on the initiative helped us secure the Executive sponsorship and focus so critical to the success of our customer centricity journey.
Beyond our on-line satisfaction/loyalty survey, we also conduct interviews to benchmark and track satisfaction with elements of the customer journey (service, support, product, account exec, etc) as we as key drivers behind the satisfaction with these touch points. For example, under support, we measure both the importance and satisfaction levels of:
As stated by others, NPS is a very valuable high-level metric; but we need to dive deeper to comprehensively understand the customer experience and set priorities for our customer experience journey.
Best regards
Roger
>> "NPS is a very valuable high-level metric; but we need to dive deeper to comprehensively understand the customer experience and set priorities for our customer experience journey."
You said it, Roger!
Are other folks using the NPS (or satisfaction) vs. importance approach to prioritize activities? Or are you using other models or frameworks to go from insight to action?
Hi folks,
The NPS conversation continues complete with interesting tidbits on how to turn insight into action in another Forrester Community: http://community.forrester.com/message/12039#12039
I see a lot of synergy between the conversations taking place in Customer Experience, Customer Intelligence and Market Insights.
It makes me think ... should some of these Communities be combined, or should the mandates be more clearly defined?
Best regards
Roger
To your point re: convergence of communities - based on our client experiences and, market observations, we're seeing growing demand on "operationalizing" a combination of predictive, perscriptive analytics and transaction data to influence customer experience - across lifecycle and channel. That's a mouthful to say our customers (e.g. financial, industrial, CPG, travel and entertainment) are asking the tough questions of "what data/insight really makes a difference?" and, "if I know what data/insight makes a difference, what's the best way to act on it?" To this end, I see a point of convergence for these communities. So, for example, rather than saying "is Net Promoter Score" the ultimate question? We ask if we capture NPS, can we translate that "insight" into customer value and corporate profitability? If we have the optino to capture NPS and, pick the other half-dozen metrics/scores, which one should we use? Its an interesting exercise and, an area where a cross-section of the communities could weigh in.
My two cents on a Friday ...
As a member of the Customer Intelligence community, I totally agree with most of the aspects which are mentioned here. Especially, I agree with Rogers statement "NPS is a very valuable high-level metric; but we need to dive deeper to comprehensively understand the customer experience and set priorities for our customer experience journey." But please let me add one aspect from the NPS strategy that has not been addressed so far.
As a big organization, we have implemented the NPS metrics in our entities almost everywhere around the globe. However, to me, implementation of NPS was never connected with terms like "metrics", "measurement" or "survey" in the first place. It is more a vehicle for a change management process. Actually, we have 2 NPS systems in place:
a) the "bottom-up NPS": a huge number of customer contacts are followed-up by a NPS call (the "first call"), separated by customer touch points. The feedback of this call is directly given back to the contact person (or his/her team), and the contact person has the chance to get back to the customer for a deeper evaluation (second NPS call). So, we have installed a NPS-based customer feedback loop process to drive the organization acting more customer-centric. In the beginning, the NPS results from the first call were solely kept inside the department of the customer contact (a sales or service team), they were not reported towards or aggregated on a higher level (so, NPS is not market research at all). Meanwhile, we started to collect the information centrally to be able to learn from the achievements.
b) the "Top-down NPS": a short, representative NPS survey, for benchmarking purposes against peers only.
So, to me the core idea of NPS has nothing to do with research. Is Net promoter the ultimate question? To be a bit provocative, I would say that this question is of minor importance. From a market research perspective, you would always find good reasons why one single metrics is not sufficient to explain or predict customer behavior. But this is not the point. To really change an organization, to really create impact, it's more important to have a concept based on an idea that is easy to understand and convincing as well on the C-level as for normal employees. Willingness to recommend? Satisfaction? Re-purchase intention? A combination of all of them? They probably all would work, as long as you are able to make the concept "thrilling" enough to make a 200.000 employee organization following it.
Any thoughts on that?
>> To really change an organization, to really create impact, it's more important to have a concept based on an idea that is easy to understand and convincing as well on the C-level as for normal employees.
I think this argument puts NPS in its appropriate context, and it mirrors what I tell clients every day. NPS can serve as an effective 'north star' or 'rallying cry' (pick your cliche) for the reasons described in this discussion, even if it's predictive value is questionable. The point is not to have good stats. It's to have alignment.
Thanks for continuing this productive conversation.
I think you're also picking up on a very important distinction which is customer experience measurement vs. voice of the customer programs and associated closed-loop response systems. I'd argue that part of what you derived value from was simply asking customers for feedback about their experience and putting into place some recovery mechanism when the answer wasn't what you'd hoped it would be. That can - and should - happen regardless of which metric is actually triggering the response (NPS, Sat, etc). You can calculate an NPS metric from that data, it's true. But the customer feedback and associated action is more valuable than the score number alone.
I have heard from many clients, as well, that asking the NPS question at this transactional level can yield very different results from NPS data collected at the brand level only. That has led them to spend a tremendous amount of time trying to understand the difference between the two data sets. My theory is that customers' perception of an experience changes over time so that once the most recent experience blends into the mix of all the other interactions they've had they may feel differently than they did immediately post-call, -visit, etc and thus respond differently to the same question. I hate to see companies spend so much time dissecting and explaining the numbers that they miss the overarching spirit and message of the feedback that comes with them.
Megan,
I think you're right regarding two aspects
A. we also see differences between the transactional NPS (bottom-up) and the NPS on brand level ((top-down). This is not very surprising, since the transactional results are based on a very recent concrete experience, whereas the representative sample on the brand level is more driven by a general perception of a provider and its performance. Having NPS developments going in different directions sometimes leads to confusions, however we have been able to figure that out by showing that both NPS movements are correlated with a time shift. A (e.g.) constantly positive development on the transactional level usually leads into a positive development on the brand level after a while.
B. yes, customers perception of experiences changes over time. Even more, different kind of transactions have a different "speed of being forgotten". This is an very important factor to be considered if you have discussion on the best company actions to take to improve the NPS (or other indicators).
I'm not sure I would classify it as the ultimate question but I think it can be an important one. The key is to understand how it relates actual customer behavior and use it (or not) accordingly. It can be a powerful tool in the arsenal but doesn't work everywhere or by itself.
Thanks for raising an interesting question about an interesting question,
Tabitha
All that matters to me is that you find out what you do good and what you do bad with your customers at every interaction. From there you promote or fix. I believe several questions can get you there, so I recommend choosing the one that works best for your business.
Well said. It's about improving, not tracking stats.
Re: the comments above: I'm also seeing several roles converge on the broad goal of better using customer feedback and other data, not just here in our communities but also in our clients' organizations. I'm pretty encouraged by the trend, even if most companies aren't there yet.
What unique pieces of value come from this kind of collaboration/enterprise view? I've been pushing some of the EFM vendors to answer this question lately, even just around bringing together all of the feedback data available across an organization. I'd love to hear your thoughts.
The point of a customer satisfaction survey is to give you accurate, actionable information. Relative to this, Reichheld, in some industries, fails. In some industries he doesn’t. There are many industries,even by Reichheld’s admission, that aren’t right for the Net Promoter question.
The best way to ensure accurate, actionable survey data is,as NPS does, to keep the survey simple. But you must vet your survey to ensure that the question or questions you ask are exactly tailored to your audience and your company.
The suggestion that accuracy isn’t important is irrational from a business perspective. As Drucker pithily and correctly said, you can’t manage what you can’t measure. And what’s the point of a satisfaction survey if it doesn’t lead to better management?
We all know intuitively that the quality of a company's customer experience affects customers' likelihood to recommend the company (and thus NPS). Now some new Forrester consumer data proves this out statistically for banks and retailers.
Harley Manning summarized the results nicely in his blog post here: Want To Raise Your Net Promoter Score? Try Improving Your Customer Experience. I'll quickly share two of the big points Harley made in his post:
1) Here's why most customer experience folks say that they use NPS, highlighting a theme that we've seen in this community discussion as well: "In brief, they use it as a simple, easy-to-understand metric — one number — for aligning the business. Its main appeal is that busy executives don’t need to spend hours studying tables and spreadsheets to get a sense of how their firms are doing. Similarly, frontline employees down to the lowest levels of the organization find that NPS makes intuitive sense."
2) The correlation for both industries is high, and that makes sense: " The correlation coefficient for CxPi and NPS for banks was 0.679, and for retailers, it was 0.618. For practical purposes, that's about as good as it gets... Of course if you are a math wonk, you’ll point out that just because two things correlate doesn't mean that one thing causes the other... However, when we look at the possible real-world explanations for these findings, we conclude that more often than not an improvement in customer experience will drive higher NPS. Other explanations, like the idea that people randomly decide to recommend a firm and then retroactively change their perceptions, aren’t ones that you’d want to put in front of your board of directors."
What do you think? Is the relationship between NPS and customer experience surprising? How about the justification executives give us for using NPS -- does that resonate?
Hi Andrew,
This is not at all surprising. I've made no secret of the fact that I'm a big fan of NPS. Harley's points highlighted in your post explain why very nicely! I'm NPS certified from Satmetrix and one of the most important takeaways from the certification course was: don't focus on the metric, focus on improving the customer experience - the metric will take care of itself.
I've heard it in theory, and we're seeing it happen in our own customer experience journey.
Best regards
Roger
Thank you all for this most interesting discussion. I was wondering if anyone has experience with POSE analysis. With this tool you can both determine the positioning of brand as well as determine the strength or favorability of the brand in comparison to another brand. I can assume such measurement could be used as part of a multi-item scale for satisfaction, attachment, and/or loyalty. But the advantage then is that it also provides insight in what aspect of the brand one is performing well or not so well and making the comparison (namely a competitive brand) explicit.
Any thoughts?
NPS is a good measure, but key is to know more than just intent. Not 'would' you recommend, but rather 'have you recommended'.
Net promoter is a good measure but not the Ultimate question. It is much better than the old Customer Satisfaction questions. For us it is also important as it captures the emotional side of a Customer Experience that gets lost in Customer Satisfaction. In fact in the research we conducted with London Business School in my third book the DNA of Customer Experience: How emotions drive value Palgrave McMillian 2007 we can statistically prove that emotions drive Net promoter. If you recommend someone you are putting some 'skin in the game'. Finally one of the best aspects of the introduction has been the debate it has generated and the ability to compare yourselves against other people.
The inclusion of emotion (head as well as heart) is so important. When you recommend, you "co-brand" your own reputation with that of the company that you are recommending. People won't do that if they believe the company will deliver a bad experience as it will backfire on the recommender's personal reputation.
Forrester Research, Inc. is an independent research company that provides pragmatic and forward-thinking advice to global leaders in business and technology.
Forrester supports leaders in 19 roles across three distinct groups: IT, Marketing & Strategy, and Technology Industry.
Aligned to your professional role, Forrester's analysts are experts in the specific technologies, issues, and trends currently impacting your business.
Fresh thinking and collaborative problem-solving through an unmatched combination of peer networking, forward-looking analysis, and professional guidance.
Our expert analysts apply custom research-based solutions and data-rich insight to your critical challenges and opportunities.