Order effects on questionnaires

A considerable body of evidence has shown that the order in which questions are asked can influence responses.

For example, if a survey has a mix of open and close-ended questions, placing the closed-ended questions first can bias responses to subsequent open-ended questions toward those mentioned in the close-ended questions.

More than 50 years ago researchers Herbert Hyman and Paul Sheatsley asked study participants two questions:

  • Communist reporter: Should the U.S. allow communist reporters from other countries and allow them to return to their newspapers and report the news as they saw it?
  • American reporter: Should a communist country like Russia should let American newspaper reporters come in and be sent back to the U.S. to report the news as they saw it?

When the American reporter question was posed first, respondents were much more likely to allow the Russian reporter access. And respondents were far less likely to expect that the Americans would have access to Russia when the communist reporter question was asked first. When the American reporter question is asked first, people are encouraged to think in terms of reciprocity. If Americans are granted access to Russia, then it’s only right for America to grant access to the Russians. But when the Russian reporter question is asked first, the pressure is to be consistent in answering the question about the American reporter (Schuman and Presser, 1981).

 

 

Leave a comment