Common Measurement Mistakes: Volume 3
Welcome to Volume 3 in Common Measurement Mistakes compendium. Missed the first two volumes? Read Volume 1 here and Volume 2 here. You should also check out previous posts such as Confusing Causation and Correlation and Confusing Measurement with Feedback, which are also common mishaps when measuring the customer experience.
Here are three more common mistakes.
Common Measurement Mistake: Sampling Problems
Sampling works well when sampling is done correctly. Sample selection and sample size are critical to creating a credible, reliable, accurate, precise, and predictive methodology. Sampling is a science in and of itself. You need samples representative of the larger population that are randomly selected.
Common Measurement Mistake: Faulty Math
Taking a binary approach to measuring satisfaction – in effect, asking whether a customer is or is not satisfied – leads to a very simplistic and inaccurate measurement.
Intelligence is not binary. People are not just smart or stupid. People are not just tall or short. Customers are not just satisfied or dissatisfied. “Yes” and “no” do not accurately explain or define levels or nuances of customer satisfaction. The degree of satisfaction with the experience is what determines the customer’s level of loyalty and positive word of mouth.
Claiming 97% of your customers are satisfied certainly makes for a catchy marketing slogan but is far from a metric you can use to manage your business forward.
If you cannot trust and use the results, why do the research?
Common Measuring Mistake: Keep it Simple – Too Simple
The “keep it simple” approach does not work for measuring customer satisfaction (or, really, for measuring anything regarding customer attitudes and behaviors.)
Customers are complex individuals who make decisions based on a number of criteria, most rational, some less so. Asking three or four questions does not create a usable metric or help to develop actionable intelligence. Still, many companies take this approach and make major strategic decisions – and often compensate their executives – based on a limited and therefore flawed approach to measurement.
Great managers do not make decisions based on hunches or limited data; “directionally accurate” is simply not good enough when our companies and our customers are at stake.
Great managers also pay attention.
In the last installment of this series we will delve into the final measurement mistake: Measurement by Proxy.