When we ask a Survey question, we assume the answer is given based on it being understood.

Words matter!

Language, terminology and cultural sensitivity can impact Survey outcome.

For example, Shanghai Municipal Tourism Administration ordered Marriott to shut down its Chinese website for a week after listing Tibet, among other parts of China, as an independent country in a email survey.

And when a question is misunderstood, the survey result is skewed for everyone, not just the individual answer.

So it's worth spending some time thinking about how to ask your question. It can mean the difference between a poor outcome and a survey result you can rely on.

 Surveys are like football, everyone in the nation consider themselves a genius on the matter.

Tony D'Andrea, Senior insight strategist, Cars.com

All too often we've seen polls and surveys that look confusing. Take this recent poll on social media, intended to raise cycle safety awareness:

Which in your opinion is more dangerous?
• o Bike running red light
o Car parked in cycle lane

The poll generated a robust response but not in the way intended. Even though awareness was the intent, being misunderstood never ends well.

The poll had 2 main concerns from our Survey practitioner perspective. First it has False Equivalence, essentially comparing things of vastly differing magnitudes. Second, the poll is a False Dilemma, as other options are available but are not presented. This can trigger a response with poor outcomes.

So how do we frame this question so that both the awareness aim and a reliable survey result is achieved?

In terms of the False dilemma issue, our subconscious can see there are at least 4 options but we are presented with only 2. Other options to consider would be:

• -Bike running red light
• -Car running red light
• -Car parked in cycle lane
• -Bike parked in cycle lane

But it's also False Equivalence, so a better way to ask the question would be to present as 2 separate questions. It was notable in the Poll comments that several people suggested that option! Now consider the first question:

Which in your opinion is more dangerous?
• o Car running red light
• o Bike running red light

But this is still not quite right. A valid option is to consider both are equally dangerous and that the individual simply does not know. Adding a Comment question will also catch options you may not have considered.

And how we phrase a question matters. For example, if we add "Thinking about" to the question, the individual is more likely to reflect upon the experience for longer. Here's the question

Thinking about running a red light, which in your opinion is more dangerous? (select one)
• o Bike running a red light
• o Car running a red light
• o Both equally dangerous
• o Don't know

  Tell us what you think
  

  Now we have a positive awareness outcome and a Survey result we can rely on.

  Survey results
  • x% said Car running red light is more dangerous
  • y% said Bike running red light is more dangerous
  • z% said they were equally dangerous
  • w% said they did not know

  Quick wins for Sensorpro customers

  1. Send our support team your survey, we'll review it for you.

  2. Chose the most relevant question style from the library of 25+ question types.

  3. Check the template library for pre-configured surveys you can use right away.

  4. If you have teams contributing to survey content and deployment, consider using our Question approvals and role based-security tools.