A Comparison of Different Approaches to Examining Whether Interviewer Effects Tend to Vary Across Different Subgroups of Respondents
It is common knowledge in the field of survey methodology that interviewers in face- to-face or telephone interviews can have undesirable effects on the obtained answers. Interviewers may introduce these effects in an active way, for example by asking suggestive questions. The effects may also arise in a passive way, if certain interviewer characteristics ultimately elicit socially desirable answers.
These active and passive effects may be systematic across all interviewers, resulting in "pure" interviewer bias, or they may differ from interviewer to interviewer, resulting in additional variance in the data that is attributable to systematic differences between interviewers rather than differences between respondents. In this chapter, we focus on the latter type of interviewer effects. One can analyze interviewer effects of this type by decomposing the variance in a (substantive) variable into variance that can be explained by the differences between interviewers, termed "between-interviewer variance," and residual or "within-interviewer" variance. Between-interviewer variance reduces the statistical precision of survey estimates, and in some cases, this variance can seriously affect data quality.
There is a long tradition of analyzing interviewer effects by means of interviewer variance analysis (for two reviews, see Schaeffer, Dykema, and Maynard 2010 and West and Blom 2017). However, very few papers have examined whether interviewer effects are stronger or weaker in specific subgroups of respondents. One might argue, for example, that cognitive difficulties, presumably experienced to a greater extent by lower-educated respondents (KnaUper, et al. 1997; Krosnick and Alwin 1987; Narayan and Krosnick 1996), obstruct the paradigmatic question-response process and provoke additional interventions of the interviewers. In this situation, the interviewers' task to make the question-response process go smoothly becomes more complex and the risk of interviewers influencing respondents' answers in a systematic way increases. Following this reasoning, the interviewer-respondent interaction is considered as a stepping stone to studying the link between interviewer effects and respondents' education level, and one can expect that interviewer effects are likely to be stronger among lower-educated respondents. Identifying groups of respondents who are more prone to interviewer effects allows for a more focused analysis of interviewer behavior, and can provide useful information to develop interviewer training programs with special attention to respondent groups that tend to engender higher interviewer effects.
We discuss the basic statistical model used in the research tradition of analyzing interviewer effects in the next section. We will argue that this basic model is not suitable for comparing interviewer effects across different respondent groups. In this chapter, we show how one can extend the basic model to allow for such investigations. In particular, we will demonstrate that lower-educated respondents elicit larger interviewer effects.