Ming, Joy, Sharon Heung, Shiri Azenkot, and Aditya Vashistha. 2021. “Accept or Address? Researchers’ Perspectives on Response Bias in Accessibility Research.” Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility (New York, NY, USA), ASSETS ’21, October 17, 1–13. https://doi.org/10.1145/3441852.3471216.

Notes

• The response biases mentioned here seems very relevant to the other forms of ICTD research also outside accessibility

In-text annotations

"“classic” defnitions of response bias from the literature, like when participants gave responses that supported the researchers’ goals (i.e. demand characteristics [53]) or did not ofend them (i.e., social desirability bias [26]) to encourage the researchers to “help” people with disabilities" (Page 2)

"One way study design could afect participants’ responses is through study fatigue [58], where participants are burdened by the length of the survey, density of information, or consecutive participation" (Page 2)

"Study design also includes task completion bias, where participants feel more inclined to fnish the tasks that they are assigned [40] or incentive bias where participants may give more positive feedback to get compensation [32, 59]. Moreover, participants in technology-based studies might experience a novelty bias or halo efect, infating feedback because they are simply excited to try something new" (Page 2)

"Prior research has found that the “social distance” between researchers and participants strongly infuence response bias" (Page 2)

"For example, a study showed that participants were more likely to give overly positive responses to a foreign researcher than to a local one because of the large diferences in social and demographic factors, even if the participants themselves were not consciously doing so" (Page 2)

"The researchers in our study referred to response bias as a “systematic skew in responses”" (Page 5)

"According to the researchers, the participants’ comfort levels depended on how they viewed themselves and their abilities, especially in relation to the researcher. Some participants withheld their feedback due to the power dynamic created by the “charity model of disability” (R13), which put the researchers in the position of “helping” the participants with disabilities. This caused the participants to respond with gratitude or attempts to not ofend the researchers." (Page 5)

"One attitude was that “everything is biased...because it goes through a human” (R15). Others echoed this sentiment and added that they “embrace[d] our humanity and biases” (R7), taking on a more cultural anthropologist perspective in exploring biases instead of immediately eliminating them." (Page 8)

"R18 explained that they take responses at face value because it’s their “job to believe what participants are saying.” Moreover, R19 was concerned that identifying response bias implied that the researchers knew better than the participants, which could reinforce power diferentials and structural oppression:" (Page 8)