Measuring Customer Experience and Loyalty: Is there “One True Method”?

In this blog post, our Chief Methodologist introduces the prevailing methods for measuring the customer experience and loyalty for services, including SERVQUAL, the Customer Satisfaction Index (CSI), the Net Promoter Score, and a relatively new approach, the Wallet Allocation Rule.

Rockbridge helps many of its clients improve customer satisfaction and loyalty by providing continuous tracking of the customer experience. When we first consult with these clients, they usually ask what measurement approach we use and how it compares to those used by others in our industry. Their interest makes it useful to provide a review of the prevailing methods used in the services sector (which includes large and small businesses and associations). I will discuss Rockbridge’s approach, with the caveat that we are not zealots for a single method.

The number of customer experience and loyalty methods nearly equals the number of consultants offering them, but a survey of the literature and best practices would reveal three prevailing approaches for quantifying the customer experience and loyalty – SERVQUAL, the Customer Satisfaction Index (CSI), and the Net Promoter Score (NPS). Other methodologies bear strong similarities to one of these three keystone methodologies. I will review these, and also talk about a relatively new approach, the Wallet Allocation Rule.Comparison-of-Satisfaction-Measurements

SERVQUAL – This methodology was developed by Parasuraman, Zeithaml and Berry as a way of quantifying the quality of a service, as opposed to a tangible product. This measurement approach gathers perceptions of performance and expectation on a range of dimensions that define the customer experience. The classic SERVQUAL approach includes rating attributes around the themes of reliability, responsiveness, assurance, empathy and tangibles. For each attribute, customers provide ratings of (a) how well a provider performs, and (b) the level of performance that is expected. For example, a customer might rate an attribute such as “employees care about your needs” a 5 on a 10 point scale, but consider 8 to be “excellent”, leaving a 3 point gap that the provider should strive to close.

Rockbridge uses a similar methodology as part of its “Path to Excellence” framework for improving customer loyalty. Rather than use the classic SERVQUAL attributes published in the literature, we tailor the attributes to fit the particular business of our client and rely on what is learned through qualitative voice-of-the-customer research. We use the same type of scaling as SERVQUAL, capturing both performance and an excellence standard, which proves useful in setting improvement goals. By knowing what customers expect, our client can avoid over-delivering. At the same time, a client may realize that customer expectation far exceeds the industry performance level, allowing for goals to align with customer need rather than matching competitors in a race to the bottom.

The multi-attribute approach used in SERVQUAL provides a clear path for managers on where to focus their resources. We provide a scorecard that shows the overall performance gap for the customer experience as a whole, and for sub-dimensions (e.g., service reliability). In sum, SERVQUAL provides a picture based on a map of customer needs and clearly identifies what to do next by revealing the largest gaps.

Customer Satisfaction Index (CSI) – Claes Fornell developed this methodology initially to set universal satisfaction benchmarks across different industries in Sweden. Today, the CSI is tracked across a wide variety of sectors all over the world. It is also widely used in the federal government sector in the U.S., where its use – for good or bad – is mandated by agencies. The CSI measures customer satisfaction by combining multiple survey measures, providing an index that is calibrated to a 0 to 100 range. The survey items tend to be shades of grey around a common construct, namely, satisfaction with the service experience, closeness to the ideal experience, and degree of meeting or exceeding expectations. The idea behind the CSI measure is that by combining these similar measures into a single number, we achieve greater reliability, meaning results are less subject to random variation due to noise in the metric. An organization relying on the CSI (or a similar measure that combines multiple items) would realize far greater stability in tracking than if it focused on a single survey question. We have found that an index made up of multiple items typically has a higher correlation with behavioral measures (e.g., purchases) than a single survey question such as “how satisfied are you”.

For Rockbridge clients, we create a similar type of index from multiple measures, providing an overarching measure of customer loyalty. Such a measure is important, over and above a more granular approach such as SERVQUAL, because it is influenced by factors outside the detailed experience ratings in a survey. These external factors may include publicity events, brand perceptions and factors beyond our client’s control (e.g., a new tax on a service). The index also offers the same “psychometric” advantages of reliability and validity.

Net Promoter Score (NPS) – Fred Reicheld introduced this metric in a Harvard Business Review article titled “The One Number You Need to Know”. The methodology relies on a single question that asks the respondent to rate their willingness to recommend the company on a scale of 0 to 10. Those who rate a company a 9 or 10 are classified as “Promoters” and those who rate their intent a 0 to 6 are classified as “Detractors”. A Net Promoter Score is computed by subtracting the “Detractors” from the “Promoters,” resulting in a score from -100 to +100. The methodology has gained considerable popularity, and stirred some controversy. Its simplicity gives senior managers a singular focus that energizes an organization to improve. The popularity of the approach results in a wide availability of benchmarks. The major complaint when a company focuses excessively on this one number is that the research fails to reveal a path for improving it, in other words, it tells you “how much” but not “why.”

The controversy I mentioned stems from what may have been exaggerated claims about the superiority of the metric to predict company financial performance. Peer reviewed research has shown that the willingness to recommend measure is comparable to other common survey metrics in predictive ability. NPS is not the superweapon that proponents have claimed, but the momentum it gained among executives, fueled by the author’s claims, results in an obsession with this one number.

Rockbridge includes a question on willingness to recommend in every customer satisfaction and loyalty survey we design, and we try to stick with the same 0 to 10 or 1 to 10 scale recommended by NPS proponents to allow comparability with the original metric, giving our clients the option of computing their own NPS. The “recommend” question is very useful in creating a multi-item loyalty index for our clients. We have also found that it is one of the best predictors of behavioral measures. Interestingly, our modeling reveals a unique problem with the NPS measure: while the 0 to 10 intent to recommend tends to work well in predicting behavior, the conversion to a net score (subtracting low raters from high raters) over-engineers the measure resulting in lower reliability and validity.

Wallet Allocation Rule – Keiningham, Aksoy and Williams present this methodology in a recently released and well-written book by the same title. This intriguing metric is computed based on rankings of ratings among all the brands a customer uses. Based on research by the authors, it is a strong correlate of share of wallet and market share because it calibrates results against available choices. The calculations can be based on a conventional measure of customer satisfaction, or other common survey metrics such as willingness to recommend or future purchase intent. The only potential change to a survey consists of gathering ratings for all service brands the customer uses. We plan to incorporate this in future client studies as appropriate, and will assess the usefulness and validity of the metric. The obvious caveat with this approach, and one I am sure the authors would agree, is that a single number is not sufficient to identify how to improve performance in meeting customer needs. The “Share of Wallet” metric should be used in conjunction with other information that leads the company along the “Path to Excellence.”

I have provided a summary of the approaches discussed above in a table. Each has its own advantages in terms of being actionable, reliable and valid. In our Path to Excellence approach to developing a measurement system, we recommend a combination of an overarching loyalty index that is based on multiple broad-based ratings, and a detailed and highly actionable goal-based scorecard rooted in SERVQUAL.

Some consultants and executives are zealots for a particular methodology. I view methodologies the same way as I do religion. I belong to a particular faith that satisfies my spiritual needs, but when I am invited by a friend to a service for another faith, I always take away a moral or spiritual insight that helps me function better in life. In the same way, I will advocate particular methods and metrics that prove to be effective for service sector companies and associations, but I make sure to rely on multiple approaches that allow a good balance between goal setting and action planning, and welcome new or different ideas that improve the chances of success. In the end, it’s not about using the “one true metric” but about identifying ways to improve customers’ happiness.

Written by: Charles Colby, Chief Methodologist