Log in / Register
Home arrow Environment arrow Research Methods in Anthropology: Qualitative and Quantitative Approaches

Retrieving the Answer Key to a Test

If there is a single culture—if there is one major factor representing knowledge of the domain and if there are no negative scores on that factor—then informant scores on the first factor represent their knowledge about—their competence in—the domain. And because of the relationship between formulas 16.1 and 16.3—that is, the relation between agreement and knowledge—we can retrieve the answer key to the set of questions on the test by looking at the most common answers—that wisdom-of-crowds thing again—and by giving more weight to the answers of the high scorers on the test when the crowd is split on the answers to a particular culture.

I tested this, using UCINET (Borgatti et al. 2002) on data from a 1995 intro class in anthropology. There were 168 students in the class and the exam had 60 questions. Recall that the ratio of the first eigenvalue to the second should be at least 3-to-1 to conclude that there is a consensus. In this case, the ratio was more than 20-to-1, so there was, indeed, one big underlying factor associated with the answers that students gave to the questions on the test. Because the purpose of the test was to gauge students’ knowledge of the material, the factor can be interpreted as knowledge, and each student’s score on that big factor can be interpreted as his or her score on the test.

Figure 16.21 shows the correlation between the first factor score for each student and the score that each student actually got on the test. The correlation is 0.96—almost perfect. It’s so close, in fact, that had I lost the answer key, I could have given every student his or her factor score as the grade for the test with no impact on the final grade for the course. And this is no fluke: Borgatti and Carboni (2007:458) ran this analysis on a test of 91 students in an organizational behavior class and got a 0.95 correlation—again, almost perfect.

The answer key for my test that was derived from the analysis was also almost perfect. Taking the majority answers to the 60 questions and adjusting for guessing, the analysis got 58 right. Here are the two questions that it missed:

‘‘Natural selection’’ selects for:

  • (1) reproductive success.
  • (2) survival of the fittest.
  • (3) survival of the species.
  • (4) adaptive radiation.
  • (5) random mutations.

FIGURE 16.21.

Plot of raw grades and scores from a cultural consensus analysis on an Intro to Anthropology test. The correlation is 0.96.

The first hominid to live in regions with cold winters was:

  • (1) Homo erectus.
  • (2) Homo hablis.
  • (3) Homo sapiens neandertalensis.
  • (4) archaic Homo sapiens.
  • (5) Australopithecus afarensis.

For the first question, 70 students picked (1), reproductive success (the correct answer), but 79 students picked (2), survival of the fittest. For the second question, 49 students picked (1), Homo erectus (the correct answer) but 90 students picked (3), Homo sapiens neandertalensis. (In those days, Neanderthals were still classified as a subspecies of modern humans. Today, some anthropologists would classify them as a separate species.) Popular culture—about Neanderthal Man and about the phrase ‘‘survival of the fittest’’—was just too powerful to be overcome fully by some anthropology lectures (box 16.6).

What this means is that if you can retrieve an etically correct answer key, you can apply the model (cautiously, always cautiously... see box 16.6) to tests of emic data—like people’s ideas about who hangs out with whom in an organization or what people think are good ways to cure a cold, avoid getting AIDS, take care of a baby, etc. It still takes knowledge of the local culture to fully understand the distribution of knowledge about a cultural domain. But when we ask people during fieldwork to tell us the uses of various plants or to list the sacred sites in a village or to rate the social status of others in a community, we don’t want to know only their opinions. We want to know the uses of the

BOX 16.6


Hruschka et al. (2008) used frame elicitation in interviews with women in Mat- lab, Bangladesh, about the causes and symptoms of postpartum hemorrhage. Of the 149 in their sample, 98 were lay women, 37 were traditional birth attendants (TBAs), and 14 were skilled birth attendants (SBAs)—that is, women who had been trained in modern medical techniques for midwifery. When Hruschka et al. ran the data through consensus analysis, the ratio of the first to the second eigenvalue was nearly 6-to-1 and there were no negative competencies. This indicated a single cultural model. But when they picked the data apart, none of the SBAs agreed with the statement “alga (evil spirits) is a cause of excessive, life-threatening bleeding,'' whereas 84% of TBAs and 78% of lay women agreed. This turned out to indicate a pattern: The SBAs agreed among themselves about many of the items in the cultural knowledge test; the lay women and the TBAs agreed among themselves; and the two groups disagreed. In other words, the two groups of women were drawing their answers to test questions from different answer keys. Hruschka et al. point out that ''anthropologists have long observed this possibility in modern society,'' citing work by Margaret Mead (1940) and Fredrik Barth (2002).

plants and the location of the sacred sites and the social status of people. We never had an answer key to tell whether informants were reporting this information accurately. Now we do.

Found a mistake? Please highlight the word and press Shift + Enter  
< Prev   CONTENTS   Next >
Business & Finance
Computer Science
Language & Literature
Political science