While revisiting industry trends in customer satisfaction surveys, we came across this rather old, but timeless article from Forbes called “Do You Trust Your Customer Survey Results?” Many things that the article talked about are relevant to both customer satisfaction survey companies and to companies that requisition these surveys. The article focuses on best practices of companies that are successful at customer satisfaction surveys. Some practices include:

• Frequently surveying the clients
• Being consistent in the survey methods
• Tracking participation
• Preventing bias
• Breaking down survey results, and
• Validating survey results

If you haven’t read our other articles about customer satisfaction surveys, you may want to also read:

• How to conduct market research

However, in this article, we focus on an area that even the Forbes article stressed on — preventing bias — about analyzing the survey data and interpreting survey results.

Why is interpreting survey data without bias important?

A paper published on the website of Berkley University identifies many factors that result in bias while interpreting and using research results. Two of these are:

• Personal prejudices and interests of those commissioning a research
• Over-compensation as a result of recognizing the personal preferences can skew results in the opposite direction

It would be wrong to state that bias only exists in analysis of research findings. Bias finds its way at every stage: selection of survey sample, defining the demographics, selection of the survey method, etc. However, even if all of these were objective and devoid of bias, the entire survey could prove to be futile if bias is not eliminated from its analysis and interpretation.

Every business that invests the time and effort in customer satisfaction has the best of intentions –to ensure customer delight, take corrective measures to maximize customer retention, deliver the value customers expect, innovate and offer new products and services.

However, most businesses expect to find some information or data to back their assumptions and beliefs. Instead of approaching customer satisfaction surveys as a means to finding what customers want, many businesses make the mistake of using this as a way of justifying actions and decisions they want to make.

This often results in a bias while analyzing and interpreting the survey results. When decisions based on biased interpretation prove to be ineffective, the tendency is to question the effectiveness of customer satisfaction surveys as a channel and this fuels a distrust in future decisions based on the findings of surveys and market research.

How to eliminate bias while interpreting survey data?

If you have completed your survey, gathered the data and are now worried about bias, it’s already too late. Eliminating bias in analysis should start while planning the survey. While designing the survey, if measures are not taken to protect research findings from biased interpretation, it is difficult to remove it during interpretation.

Here are some things to address in order to expel research bias:

• Avoid Design Bias: The survey should not be designed around “who you want to survey” but around “who will provide the most reliable responses”. For example, a survey around pet-food needs to be done among pet-owners. On the other hand, social impact of a pet should include pet owners and people who live near pet owners.
• Participant Bias: It is important to select participants in the survey who do not have a strong bias — positive or negative. There are many ways to do this: a) not letting participants know which company the survey is for; b) safeguarding participant anonymity; c) shielding participants from being influenced — intentionally or otherwise — by the company conducting the survey, etc.
• Surveyor Bias: Many times, the person conducting the survey knowingly or unknowingly influences the survey, resulting in a bias. It is important to make sure that the survey is conducted by someone who merely asks questions without implying, suggesting or influencing the answer. The Surveyor needs to listen to the respondent in totality without contributing to the answer. Here’s an example of surveyor bias: Let’s say the survey is conducted using a rating scale where respondents are expected to provide answer on a scale  of 1 to 5 with 5 being the highest. Not all respondents are absolutely clear about the scale and individual perceptions of the scale can vary. Many times, surveyors make the mistake of suggestion—”so if you are happy, would you say a 5; or if you don’t have too many issues, would you say a 4?” These suggestions result in a bias that can skew the survey results.
• Procedural Bias: If the research is too rushed, it can result in hasty responses that aren’t accurate. On the other hand, if it is allows too long a time frame, the participants may pace it out over such an extended period that extraneous factors affect their sentiments to different questions. In turn, it can result in consistency of response and unreliable data.
• Treating Extremes: Regardless of the measures taken, there are bound to be extreme responses that aren’t representative of the audience. Before starting the research, you need to decide on how you will treat these extremes. Is your research going to discount them, provide differential weight in analysis, etc. There are pros and cons of every system, but it is important to decide early on this and not at the time of analysis and interpretation.
• Vested Interest: How well you are able to protect the research from being influenced by vested interests at every stage will determine what you find and how it is interpreted.
• Impartiality: A key to the effectiveness of the survey can lie in who plans, executes, conducts, analyses and interprets it. Impartiality of the people involved in the survey is important to how it is done, and how it is interpreted. This can influence a survey at every stage and to ensure accurate data and meaningful interpretation, it is also important to ensure the impartiality of the people behind the survey.

Customer Satisfaction Surveys from HMS

This is an article by a customer satisfaction company about eliminating bias in surveys. So how can this not be a biased article?

We do not deny the bias in this article. However, we do strongly believe that you need to remove bias from your surveys in order to get the most out of them. Outsourcing the survey to an outside industry service provider will help eliminate the bias.

Get the best from your customer satisfaction surveys by eliminating bias by outsourcing every aspect of the survey. Let HMS “Partner for Success” with your business by developing and fulfilling your customer satisfaction survey needs.