Avoid Long Questionnaires?1111
-
For many researchers, designing a short questionnaire can be more challenging than creating a long one. Whenever a survey is needed, every member of the research team tends to want to add more questions to gather as much information as possible. Each proposed section seems crucial, and every question appears indispensable. Even unrelated topics sometimes get included in the survey. As a result, the number of questions increases, and the questionnaire becomes longer and longer.
Does this scenario sound familiar? While it's understandable that researchers want to ask as many questions as possible in a single survey, unfortunately, in online surveys, lengthy questionnaires often come at the expense of data quality. Here's why:
Long Questionnaires Lead to Lower Responses and Dropouts
As the length of a questionnaire increases, both the response rate and the completion rate tend to decrease. Respondents are generally reluctant to spend too much time answering questions. If they know in advance that completing the questionnaire will take a significant amount of time, many may choose not to participate at all.
It is crucial to mention the survey's length and the estimated time required when inviting participants. Even if not explicitly stated, respondents can gauge the length of the survey through a progress bar. If they perceive the questionnaire to be too lengthy, they are likely to abandon it midway, which affects the representativeness of the sample.
In a study by Galesic and Bosnjac (2003), 3,472 respondents were randomly assigned to three groups, each participating in online surveys of different lengths, requiring 10, 20, and 30 minutes to complete, respectively. The results showed a significant decline in the number of participants who completed the questionnaire as the survey length increased.
This finding highlights the importance of keeping online surveys concise to improve completion rates and maintain data quality.
Long Questionnaires Impact Data Quality
While some respondents may tolerate lengthy questionnaires, the burden and stress caused by them can alter their response strategies. Instead of carefully considering each question to find the best answer, respondents may look for shortcuts to complete the survey with minimal effort. This behavior is known as the "satisficing strategy," where respondents do the bare minimum to fulfill the survey's requirements without exerting much effort.
As a result, you might observe patterns such as respondents choosing the first option for every question, selecting answers in a straight line for matrix questions, or picking answers at random. These behaviors degrade the data quality, rendering it less valuable for analysis.
Galesic and Bosnjac's experiment also tested how questionnaire length affects data quality. They used several indicators to measure this, including response time, item response rate (the proportion of questions completed within each survey section), the length of answers to open-ended questions, and variation in responses to matrix questions.
Among all these quality indicators, the only factor not affected by questionnaire length was the item response rate. However, it's unclear whether this was due to the survey system enforcing mandatory responses, which might have prevented the item response rate from being influenced by the survey length. All other quality indicators were negatively affected by longer questionnaires.
This highlights the importance of designing shorter surveys to maintain high data quality, ensuring that respondents remain engaged and provide thoughtful, reliable responses.
Surveys in DataRolls
At DataRolls, we carefully select best online surveys that fit your preferences, enhancing user experience and response rates. This allows you to easily earn extra money by participating in surveys.
While DataRolls does offer long online questionnaire survey, you have the option to choose shorter ones, thanks to the large variety available. This flexibility means you can find reward surveys that suit your schedule and preferences, making it convenient to participate without feeling overwhelmed by longer questionnaires.