The Two-Group Pretest-Posttest without Random Assignment
Figure 4.1c shows the design for a quasi-experiment—an experiment in which participants are not assigned randomly to the control and the experimental condition. This compromise with design purity is often the best we can do.
The classical design without randomization: The quasi-experiment.
Program evaluation research is usually quasi-experimental. Consider a program in rural Kenya in which women farmers are offered instruction on applying for bank credit to buy fertilizer. The idea is to increase corn production.
You select two villages in the district—one that gets the program, and one that doesn’t. Before the program starts, you measure the amount of credit, on average, that women in each village have applied for in the past 12 months. A year later, you measure again and find that, on average, women in the program village have applied for more agricultural credit than have their counterparts in the control village.
Campbell and Boruch (1975) show how this research design leads to problems. Suppose that the women in the program village have, on average, more land than the women in the control village have. Would you (or the agency you’re working for) be willing to bet, say, $300,000 on implementing the program across the district, in, say, 30 villages? Would you bet that it was the intervention and not some confound, like the difference in land holdings, that caused the difference in outcome between the two villages?
The way around this is to assign each woman randomly to one of the two conditions in the experiment. Then the confound would disappear—not because land holding stops being a factor in how well women respond to the opportunity to get agricultural credits, but because women who have varying amounts of land would be equally likely to be in the treatment group or in the control group. Any bias that the amount of land causes in interpreting the results of the experiment would be distributed randomly and would be equally distributed across the groups.
But people come packaged in villages, and you can’t give just some women in a small village instruction about applying for credit and not give it to others. So evaluation of these kinds of interventions are usually quasi-experiments because they have to be (Further Reading: quasi-experiments).