Methods

Data Sources

This chapter analyzes interviewer observations from two surveys - the ESS and the NSFG. The ESS is a cross-national survey of attitudes, beliefs, and behavior patterns of persons aged 15 years and older living in private households in more than 30 European nations. For this analysis, we analyzed data from the fifth wave of the ESS. The subset of ESS countries analyzed here (response rates in parentheses) included Belgium (53.4%, n = 1,704), Bulgaria (81.4%, n = 2,434), Estonia (56.2%, n = 1,793), Germany (30.5%, n = 3,031), Ireland (65.1%, n = 2,576), Slovakia (74.7%, n = 1,856), and the United Kingdom (56.3%, n = 2,422). All countries targeted the general population. Fieldwork for the ESS is centrally monitored,

Response rate = valid interviews/[total sample - (sum H, I, K, L, M)]; H = institutions, I = unoccupied, К = ineligible, L = moved, M = deceased.

and fieldwork procedures are centrally reported and documented following strict guidelines (www.europeansocialsurvey.org/data/; see Chapter 22 for additional analyses using ESS data). Computer-assisted personal interviewing (CAPI) was used in all countries except for Bulgaria and Slovakia, which each used paper-and-pencil interviewing (PAPI). Across this subset of countries, ESS teams collected data from 15,816 respondents between 2010 and 2011. After conducting an interview, the interviewer recorded post-survey observations; we recoded five of these observations capturing respondent behaviors and interviewing environments.

The NSFG is a national survey measuring sexual and reproductive health in 15- to 49-year-olds living in the United States. The survey takes approximately 60 minutes to complete, and has two sections: an in-person CAPI section, where all interviewers are female, and an audio computer-assisted self-interview (ACASI) section. Data were collected from 15,820 interviews over 12 quarters of data collection from January 2016 through December 2018 (AAPOR RR3 = 65.3%; see Lepkowski et al. (2013) for design details). After conducting an interview, the interviewer is asked to record 30 observations about the conditions surrounding the data collection, including details about the environment, the respondent's response behaviors, and the respondent's mood. We chose 22 of the 30 observations that explicitly described respondent behaviors and interviewing environments for the LCA (details below). A small number of cases (n = 52) with missing values for the interviewer observation data were excluded, resulting in a final sample size of 15,768.

Post-Survey Interviewer Observations in the ESS

In the ESS, interviewers recorded the five post-survey interviewer observations on a five- point scale (ranging from "never" to "very often"). Of these five observations, one captured the interviewing environment (an objective observation of how often someone else was present and potentially interfering during the interview). The remaining four were subjective interviewer perceptions and observations of respondent behaviors during the interview:

• • Whether the respondent asked for question clarification.
• • Whether the respondent seemed to understand the questions.
• • The respondent's reluctance to answer any questions.
• • The respondent's effort to answer the questions to the best of his/her ability.

Post-Survey Interviewer Observations in the NSFG

We analyzed 22 post-survey interviewer observations collected in the NSFG. Of these observations, the 12 objective observations of the interviewing environment included: ^[1]

• • Observations of (1) the seating arrangement (next to respondent, facing the same way; next to respondent, at a right angle; across from the respondent; other) and
• (2) the presence of others within hearing range (no one present; others present, not able to hear; others present, able to hear part of the interview; others present, able to hear the entire interview).
• • Additional reports of the respondent's ability to see the computer screen during the face-to-face CAPI section (yes, all questions; most, not all questions; a few questions; none), the interviewer's ability to see the computer screen during the ACASI section (yes, no), and in general what the interviewer's self-reported mood during the interview was (happy, neutral, or sad or unhappy).

The 10 subjective interviewer perceptions and observations of respondent behaviors included the following:

• • Observations of behaviors during ACASI, including (1) how much help the respondent needed from the interviewer (none, a little, a lot, interviewer-administered);
• (2) the respondent's use of headphones, which may help respondents with literacy problems better understand the questions being asked (at least some of the time, never); (3) the respondent's use of text and audio, both of which may help with comprehension (text only, text and audio, audio only, don't know); and (4) what support was used to hold the laptop, which could measure respondent comfort during the interview (table, lap, other).
• • Observations of difficulty using the ACASI application (any, none) and the interviewer's opinion of respondent attentiveness (not at all, some, very).
• • Three observations of the respondent's mood that may affect his/her ability to clearly think about responses to the questions: (1) whether the respondent was upset (yes, no) or (2) tired (yes, no), and (3) in general how the respondent acted during the interview (hostile or neutral, friendly).
• • An overall observation of the quality of information provided by the respondent (excellent, good, fair, or poor).

• [1] Four indicators (yes, no) of distractions: (1) whether the television was on, (2)whether the respondent received phone calls, (3) whether children were presentand needed attention, and (4) other distractions. • Observations of the interview: (1) location (on the respondent's property, in theinterviewer's car, in another public place), (2) atmosphere (chaotic, noisy; someinterruptions; ideal - quiet and calm), and (3) language (English, Spanish, both).