Common Measurement Mistakes: Volume 2
In this ForeSee Blog series, we’ve been talking about some common measurement mistakes we see across the industry. You can read Volume 1 here, and we also recommend reading some previous posts such as Confusing Causation and Correlation and Confusing Measurement with Feedback, which are also common mishaps when measuring the customer experience.
Below are a few more:
Common Measurement Mistake: Forgetting the Real Experts are Your Customers
Experts, like usability groups, have their place. But who knows customer intentions, customer needs, and customer attitudes better than actual customers? When you really want to know, go to the source. It takes more time and work, but the results are much more valuable. We cannot say how many times we’ve been in meetings with analysts and experts who swear that a new site navigation system will solve every problem on a particular website. (Oddly enough, there often is money to be made if the analysts are hired to develop that new navigation system. Unfortunately solutions are often based on what the expert can provide rather than what the customer needs.) Meanwhile, what customers want are more product varieties to choose from. Experts and consultants certainly have their place, but their advice and recommendations must be driven by customer needs as much if not more than by organizational needs.
Common Measurement Mistake: Gaming the System
Unfortunately many feedback and measurement systems create bias and inaccuracy. How? Ask the wrong people, bias their decisions, or give them incentives for participation. Measuring correctly means creating as little measurement bias as possible while generating as little measurement noise as possible.
Try to avoid incentivizing people to complete surveys, especially when there is no need. Never ask for personal data; some customers will decline to participate if only for privacy or confidentiality concerns.
Also never measure with the intent to prove a point. Unfortunately, research run by internal staff can often contain some amount of built-in bias. As employees we may, however unintentionally, create customer measurements to prove our opinions are correct or support our theories, but to what end?
Customer measurements must measure from the customers’ perspective and through the customers’ eyes, not through a lens of preconceived views.
Next week, in Volume 3 of Common Measurement Mistakes, we will discuss how Sampling Problems, Faulty Math, and Keep it Simple – Too Simple are common mistakes when measuring the customer experience.