Conducting research is important---as well as time consuming and even costly. That’s why companies conduct research infrequently, which contributes to the pressure to do it exactly right.
One mistake to avoid, if you're planning to conduct a survey, is including too many response choices for each question. Why? Research shows that respondents are less consistent in their responses when faced with “too many” choices. Inconsistency erodes the likelihood that a sample size selected to yield statistically signifcant responses, will deliver the confidence level and margin or error you are seeking.
In “Too Many Choices Can Spoil the Research,” published in the Journal of Consumer Research, ‘choice experiments’ were disguised as questionnaires that sought to learn which brands or products consumers prefer. Participants were asked to choose among a number of options (such as delivery time and quality of ingredients) for ordering pizza. Twenty two questionnaires, with varying numbers of response choices, were administered.
Although the questionnaires that displayed many different choices allowed more precise or “efficient responses,” they also increased “cognitive load” on the participants (requiring more thought for each response). This caused their responses to decrease in consistency.
How many response choices are too many? For the most consistent results, focus on providing only the response choices that are critical to your analysis. Stay focused on the primary objective of your research and resist the temptation to throw in additional responses just to ‘see what happens.’
Source:
Journal of Consumer Research
Subscribe to Brandeo’s free weekly newsletter or RSS feed