The Mismeasurement of Crime?

We have previously taken the position that microlevel theories often produce weak/moderate empirical results. This ought to cause researchers to suggest amendments to their explanations or hypotheses. After criminologists perform a test of an explanation of crime, they often find that they are presented with unexpected results. Those unexpected results are ones that challenge the nature of the explanation of crime the researcher offered and tested. When criminologists are faced with an empirical test that fails to confirm a hypothesis, they are taught to explore the reasons for that failure and often offer an ad hoc rationale for the findings. For example, there are many criminological studies that repeat the following statement: “[t]he failure to find an effect could be . . .” This ad hoc rationale is a situation that Karl Popper (1992, 2002) and others have cautioned against (Bamford 1993; Carroll 2003; Derksen 1993; Foster and Sober 1994; Pigliucci 2010). Thus rather than reject an explanation or hypothesis, criminologists often suggest that the unexpected results stem from specification or measurement problems, yet they are also often unwilling to accept that the specification error has something to do with the nature of the theory rather than its measurement.

Ad hoc explanations are warranted when there are deficiencies in translating an explanation into an appropriate empirical test. These kinds of ad hoc explanations may also be reasonable when there are clear problems related to operationalizing the independent variables, perhaps because of their abstract nature or because one can imagine other ways of measuring those variables. The search for the reasons that the research failed appears wise to the extent that it would be foolish to reject an explanation based on one empirical test, or until we can be sure that the explanatory variables are being measured correctly. But we note that when explanations of crime yield questionable results on multiple occasions, those particular explanations of crime may be problematic and require rejection due to inconsistent results.

Criminologists may also argue that it is also plausible that empirical tests of explanations of crime produce weak results because there are numerous individual-level factors that may be related to the causes of crime, and they cannot all be assessed in a single model—that is, there are so many potential individual-level causes of crime that it is hard to measure the effect of all those causes in one study. Some of the causes may also be unknown. This is likely true and returns us to the observation that models that are not correctly specified because of omitted variable bias should be interpreted with caution (Berry 1993).

Another possible response to our argument would be the need for greater attention to integrated explanations of crime—that is, if it is indeed true that there are a large number of individual-level factors that cause crime, then efforts at integrating criminological explanations of crime and testing those explanations should be able to solve this problem. However, combining weak explanations of crime is likely to produce little significant gain in explanatory power because only small gains can be achieved by combining weak explanations and, at the same time, when large numbers of variables are examined simultaneously, covariate space expands and the overlap between independent variables is likely to increase significantly, limiting the explained variance that can be achieved from such approaches.

We propose that the mismeasurement of crime is not rooted entirely in model specification; rather, the poor explanation has to do with the nature of the legal definition of crime. Rather than take this idea seriously, criminologists continue to examine existing and new modifications of traditional individual-level explanations of crime based on the assumption that the cause of crime must be individual and that the weak results empirical tests produce are due to the mismeasurement of the independent variables. Instead of examining the possibility that the dependent variable “crime” is problematic, orthodox criminology continues testing more and newer (or revised) individual explanations of crime with the most current political definition of crime (which may have changed over time) as the dependent variable. Thus over time, what happens is that the explanation of crime changes, but so too does the measure of crime. Since the political variable “crime” is not a constant over time or place, each new test of a theory assesses the new explanation with a dependent variable that includes change within its definitional structure. Little can be learned about crime when the measure of crime constantly changes. That change, we suggest, must occur over time and place because that is part of the nature of the definition of crime as a legal outcome.