PLUSES AND MINUSES OF DIRECT OBSERVATION
On balance, direct observation provides much more accurate results about behavior than do reports of behavior. Ricci et al. (1995) studied 40 people in Kalama, a peri-urban village about 15 miles north of Cairo. One day, a set of trained observers watched the 40 participants for 2.5 hours. The observers noted whether people were engaged in any of 17 activities and how long each person spent at each activity.
The next day, the participants were asked to recall, sequentially, what they had done the entire day before, and how long they had spent at each activity. The interviewers did not mention any of the 17 activities, but they tried to improve respondent recall by asking about activities before and after locally significant time markers, like call to prayer. Ten of the 40 were toddlers, so Ricci et al. focused on the recall data of the 24 adults and the six school-age children.
Ricci et al. were very forgiving in their analysis. Informants were scored as being correct if they could recall an activity at all and say correctly whether it had been done in the morning or afternoon observation period (9:00-11:30 a.m. or 12:30-3:00 p.M.). Ricci et al. scored only errors of omission (leaving out activities) and threw out errors of commission (inventing activities that had not been observed).
And informants still got it wrong—a lot. Across men and women, across agricultural and nonagricultural households, informants got it wrong, on average, 56% of the time. Five of the 6 women who had been observed breast-feeding failed to report that activity the next day; 13 of the 15 women who had been observed washing clothes failed to report that activity the next day.
If you want to know whether, say, caring for animals happens more often than, say, gathering fuel, self-reports might be enough. But if you want to know how often those behaviors actually occur, then nothing short of direct observation will do.
I don’t want to give the impression, however, that direct observation data are automatically accurate. Lots of things can clobber the accuracy of directly observed behavior. Observers may be biased by their own expectations of what they are looking for or by expectations about the behavior of women or men or any ethnic group (Kent et al. 1977; Repp et al. 1988).
You may feel awkward about walking around with a clipboard (and perhaps a stopwatch) and writing down what people are doing—or with beeping people and asking them to interrupt what they’re doing to help you get some data. This is a reasonable concern, and direct observation is not for everyone. It’s not a detached method, like sending out questionnaires and waiting for data to be delivered to your doorstep.
It is not a fun method, either. Hanging out, participating in normal daily activities with people, and writing up field notes at night is more enjoyable than monitoring and recording what people are doing.
But many fieldworkers find that direct observation allows them to address issues that are not easily studied by any other method. Grace Marquis (1990) studied a shantytown in Lima, Peru. Children in households that kept chickens were at higher risk for getting diarrhea than were other children. The chickens left feces in the homes, and the feces contained an organism that causes diarrhea. Continuous monitoring showed that children touched the chicken droppings and, inevitably, touched their mouths with their hands. It was hard, tedious work, but the payoff was serious.
Direct observation is time consuming, but random spot-checking of behavior is a very cost effective and productive way to use some of your time in any field project. When you’re studying a group that has clear boundaries (a village, a hospital, a school), you can get very fine-grained data about people’s behavior from a TA study, based on random spot checks. More importantly, as you can see from table 14.2, with proper sampling you can generalize to large populations (whole school districts, an entire aircraft manufacturing plant, even cities) from spot checks of behavior, in ways that no other method allows.
You may be concerned that a strictly observational approach to gathering data about human behavior fails to capture the meaning of data for the actors. This, too, is a legitimate concern. A classic example is Geertz’s (1973:6-7) observation that a wink can be the result of getting a speck of dust in your eye or a conscious act of conspiracy. And that’s just a wink. People can engage in any of thousands of behaviors (skipping a class, wearing a tie, having their navel pierced . . .) for many, many different reasons. Knowing the meaning of behavior to others is essential to understanding it ourselves.
On the other hand, one of our most important goals in science is to constantly challenge our own ideas about what things mean. That’s how theories develop, are knocked down, and gain in their power to explain things. Why shouldn’t we also challenge the theories—the explanations—that the people we study give us for their own behavior?
Ask people who are coming out of a church, for example, why they just spent 2 hours there. Some common responses include ‘‘to worship God,’’ ‘‘to be a better person,’’ ‘‘to teach our children good values.’’ Hardly anyone says ‘‘to dress up and look good in front of other people,’’ ‘‘to meet potential golf partners for later this Sunday afternoon,’’ or ‘‘to maximize my ability to meet potential mates whose ethnic and social backgrounds are compatible with my own.’’ Yet, we know that these last three reasons are what some people would say if they thought others wouldn’t disapprove.
Finally, you may have some qualms about the ethics of obtrusive observation. It cannot be said too often that every single data collection act in the field has an ethical component, and a fieldworker is obliged every single time to think through the ethical implications of data collection acts. Personally, I have less difficulty with the potential ethical problems of obtrusive, reactive observation than I do with any other data collection method, including participant observation. In obtrusive observation, people actually see you (or a camera) taking down their behavior, and they can ask you to stop. Nothing is hidden.
In participant observation, we try to put people at ease, make them forget we’re really listening hard to what they’re telling us, and get them to ‘‘open up.’’ We ask people to take us into their confidence, and we are handed the responsibility for not abusing that confidence.
But the method that presents the most ethical problems is unobtrusive, nonreactive direct observation.