Student Response, Missing Data, and Survey Bias

Student Response, Missing Data, and Survey Bias

A recent paper on Audience Response Systems and Missingness Trends in JMIR Formative Research identified an issue that educators need to pay attention to when using ARS to discuss sensitive issues in a school setting. As we’ve noted elsewhere in discussing the value of using ARS to facilitate active student engagement, the anonymity of ARS can help shy students overcome their reticence and diffidence. It can give a voice to students who feel uncomfortable sharing the reality of their experiences.

The Flip Side of Anonymity

But the flip side of anonymity is that it can enable students to remain silent if they choose to do so. Not clicking a clicker is even more discreet than not raising one’s hand for one side of a question or another. While it is hoped that the assurance of anonymity would enable students to speak who might not otherwise feel comfortable doing so, the promise of anonymity may not be enough to overcome a student’s reticence. Students who feel humiliated or have had an experience that has harrowed them may feel too diffident to admit to the experience even under conditions of anonymity.

Audience Response Systems Promotes Anonymity in Uncomfortable Conversations
Audience Response Systems Promotes Anonymity in Uncomfortable Conversations

Nonresponse is Response Too

And that’s a potential pitfall for educators. When students don’t answer a question, there’s a risk that an educator may gain a biased view. The absence of a response does not mean that students have no familiarity with (or experience with) the subject of the question. It just means that the response set is incomplete.
To be sure, the biases that could result from participant nonresponse would be most significant if the educator were using the ARS to gain a statistically accurate picture of student views and experience. However, if the educator is using the ARS as a way to hold a more anonymous discussion of sensitive topics – and not trying to capture research-quality response data – the risk of bias could be a moot point. Even if several students have been exposed to a sensitive issue – whether that is a sexually transmitted infection or violence in the home – and the ARS results show only one or two acknowledging it, the students who did not respond and who feel isolated because of the experience will know that they are not alone and may be more likely to seek help.

Leave a Comment

Capterra