What are the advantages and disadvantages of an online survey? (Part 2/2)

At Satisfy.io everything happens online, from creating a campaign to viewing the content by a target audience. In this article we look at the pros and cons of online research.

Following on from the first part of this article, this second part looks at the remaining advantages and disadvantages of online surveys.


The specificity of responses to online surveys

The use of online surveys would influence the nature of the data collected.


Firstly, the quality of data from online surveys would be higher than that of data from postal questionnaires due to a lower rate of non-response to closed and open-ended questions (Shin et al., 2012: 223; Denscombe, 2006: 252).


Respondents would be less likely to drop questions when the survey is on the Internet. Some studies have also shown that respondents tend to write longer responses to open-ended questions in web-based questionnaires and to be more self-disclosing than in mail-based formats (Denscombe, 2006: 252; Shaaefer and Dillman, 19984 ). Others argue instead that online surveys tend to have a bias against open-ended questions (fewer responses) (Wolf et al., 20085 ).


Responses with a social acceptability bias would also be less present in online questionnaires because of the anonymous nature of this method (Dillman et al., 2009: 3; Beck et al., 2009: 3; Bigot et al, 2010: 6), and more specifically when topics considered sensitive by respondents (such as personal problems or risk behaviours) are addressed (Wang et al., 2013: 1008; Lindhjem and Navrud 2011: 12; Ganassali, 2008: 21; Nagelhout et al., 2010: 9; Bigot et al., 2010: 39).


In a study entitled "Is Web Interviewing a Good Alternative to Telephone Interviewing? Findings From the International Tobacco Control (ITC) Survey", researcher Nagelhout notes that telephone survey respondents were generally more negative about smoking than online survey respondents.


A study comparing the results of a telephone survey and a web survey also showed significant variations in responses regarding lifestyle, health and exercise (Greene et al., 2008: 245). The study conducted by Stephenson and Crête (2011: 27) shows that, due to the more anonymous nature of online surveys, respondents would be less embarrassed to choose 'I don't know' or 'I don't want to answer'.


Finally, Frippiat and Marquis point out that: "Responding via the Internet makes individuals less sensitive to the intrusive nature of the questions than if they were responding in other ways, and it allows them to report more non-standard or socially undesirable behaviour" (2010: 63).


Quality of responses


Some studies also show that respondents to online surveys have a lower tendency to satisficing, a phenomenon that evokes the idea that a respondent, under certain circumstances (long questionnaire, lack of concentration, disinterest in the subject), distances himself from the questionnaire while reducing the effort he puts into answering correctly (Lindhjem and Navrud, 2011: 38; Ganassali, 2008: 21; Stephenson and Crête, 2011: 27). The term would translate into "low involvement of the respondent" (Frippiat and Marquis, 20106).


This phenomenon is influenced by various factors: the degree of difficulty of the task asked of an individual, the skills it requires of him or her (his or her capability) and his or her general motivation to participate (Heerwegh and Lossveldt, 2008: 837; Frippiat and Marquis, 2010: 63).


Satisficing can be observed in a large number of 'don't knows' or unanswered questions, in the repeated choice of 'mid-range' (ticking the middle box for each question that allows it), in the endorsement of the status quo as well as in the choice of the simplest or most acceptable answer without thinking (Heerwegh and Lossveldt, 2008: 838; Frippiat and Marquis, 2010: 66).


Other than through data analysis, it is possible to detect satisficing by integrating a stopwatch to know the response time. The shorter the response time, the greater the tendency to satisfy (Bigot et al., 2010: 52).


This phenomenon can be mitigated by formulating clear and concise questions (Bigot et al., 2010: 41).
Although sometimes difficult to distinguish from socially desirable responses, disengagement would be more observable in online surveys than in interviews conducted with an interviewer (Bigot et al., 2010: 11).


This can be explained by the fact that interviewers know how to capture and hold the attention of respondents while motivating them; they also promote a better understanding of the questions (Grenn and Tunstall, 19998 ; Downes-Le Guin, et al., 2012: 2; Lindhjem and Navrud, 2011: 12 ; Holbrook et al.9). Holbrook and his co-authors further claim that respondents to online surveys are frequently subjected to multitasking, which refers to the idea of doing several things at once.


Online respondents often have multiple web pages active at the same time, and this may lead to more satisficing in online surveys than in other types of surveys. Similarly, these researchers argue that completing a survey online is more difficult because it requires an understanding of computer language as well as the ability to manipulate technological equipment.
Indeed, the face-to-face survey would facilitate the response process since the questions are read orally, whereas the online survey would require more effort from individuals (Heerwegh and Lossveldt, 2008: 838).


However, other authors (Chang and Krosnicjk, 201010 ; Skitka and Sargis, 200511 ) argue the opposite and claim that online surveys would decrease the tendency to disengage; these studies confirm the need for further knowledge on this subject.
Finally, in their research comparing telephone and web survey results, Greene et al (2008: 245) found that respondents were less likely to agree with factual statement questions in the web format than in the telephone format, an observation that suggests that the results are more authentic in a web format.


At Satisfy.io, our research is done online while allowing us to reduce the phenomenon of satisficing. Indeed, the heart of Satisfyy is that the respondent does not have to write a line. The analysis of their reactions is done without any intervention on their part, which allows the respondent to concentrate on the video content.


The importance of the questionnaire format and its duration

In addition to satisficing, Bigot et al. point out that the effects of fatigue, which can be seen in 'lower response variance at the end of the questionnaire' (2010: 38), are elements that can affect the quality of the results.


They also mention that online surveys should ideally be completed within 15 to 20 minutes (2010: 38).


On a technical level, some researchers note that the presence of a visual aid integrated into the survey facilitates the understanding of the questions while allowing for nuances (Dillman et al., 2009: 16; Lindhjem and Navrud, 2011: 12; Stephenson and Crête, 2011: 27). Citing observations of significant variance in responses depending on the choice of visual, Bigot et al. state that we must still be vigilant in drawing conclusions in this regard.


Furthermore, the progressive unveiling of the questionnaire (pacing) inherent in the Web format allows the questions to be displayed progressively in the questionnaire. Unlike the paper format, where the respondent can quickly consult all the questions and choose which ones to answer or not, the web format 'forces' the respondent to answer one question at a time before being able to consult the rest of the questionnaire (Frippiat and Marquis2010: 56). Important information should also be at the top of the page rather than at the bottom, because respondents would spend more time on it (Bigot et al., 2010: 51).


Finally, progressive disclosure also has the advantage of 'limiting the effort that the respondent has to make to answer the questionnaire' since the filter questions allow the respondent to avoid answering less relevant questions as he or she answers (filter questions) (Frippiat and Marquis 2010: 56; Bigot et al., 2010: 22). For Frippiat and Marquis, these technical elements make it possible to reduce non-responses linked to the questionnaire.


The Internet survey also makes it possible to obtain data on non-responses. In the case of postal surveys, individuals who do not understand a question or who do not wish to answer all the questions do not return the questionnaire if it is incomplete. In the case of online surveys, it is possible for the surveyor to determine which questions were problematic by the number of times they were discarded or when respondents abandoned the questionnaire since this information is available. On the one hand, this information makes it possible to better assess the quality of a questionnaire and, on the other hand, to establish that 'non-responses can no longer be considered totally random, since they depend on the very variable that is the subject of the survey' (Frippiat and Marquis, 2010: 13).


Conclusion

Online surveys offer a low-cost way of investigating emerging phenomena, tackling social issues from new angles and using large samples. Compared to more traditional methods, such as telephone and mail surveys, online surveys have many advantages.


If you have any questions about setting up an online survey, please contact us: hello@satisfyy.io

Subscribe to receive articles right in your inbox