Type III Error
The extant literature on the Type III (y) error originated in statistics. Frederick Mosteller [1916-2006], one of the most eminent statisticians of the twentieth century, reported:
In other words it is possible for the null hypothesis to be false. It is also possible to reject the null hypothesis because some sample Oi has too many observations which are greater than all observations in the other samples. But the population from which some other sample say Oj is drawn is in fact the right-most population. In this case we have committed an error of the third kind. (p. 61)
This is commonly referred to as “the error associated with solving the wrong problem precisely” (Mitroff, 1998, p. 15). Type III errors normally occur during the formulation of problems, the phase in which the actual details surrounding the reported problem are exposed, validated, and verified as part of the process of problem reformulation (reformulation is where the initial reported problem statement is validated by relevant stakeholders). We denote this revised problem statement the real (or formulated) problem, to differentiate it from the reported problem. Mitroff (1998) identifies the five most common causes of a Type III error:
- 1. Picking the wrong stakeholders
- 2. Selecting too narrow a set of options
- 3. Phrasing a problem incorrectly
- 4. Setting the boundaries/scope of a problem too narrowly
- 5. Failing to think systemically.
Each of these issues is addressed in this text, with the fifth cause (and its avoidance) being the ultimate driver in writing this text.
Adams and Hester (2012) devise a medical analogy to explain the Type III error:
The systems practitioner faced with a reported problem needs to act much like a physician.
The physician listens to the symptoms reported by a patient, but does not accept the diagnosis of the patient. The physician cannot rely solely on the patient’s story and symptoms, but must gather empirical data by conducting tests, taking physiological measurements, and conducting a physical examination. The systems practitioner is in a similar professional relationship with the client that has a systems problem. Problem reformulation ensures that the scope of the problem is properly abstracted from the real-world and defined. The problem system must be adequately bounded, include empirical data of both the quantitative and qualitative types, and include an understanding of both the environment and relevant stakeholders. (p. 28)
Mitroff and Featheringham (1974) elaborate on the importance of proper problem formulation:
The initial representation or conceptualization of a problem is so crucial to its subsequent treatment that one is tempted to say that the most important as well as most difficult issue underlying the subject of problem solving is precisely ‘the problem of how to represent problems.’ (p. 383)
Failure to properly define the scope of the problem results in inadequate problem statements and is commonly referred to as “the error committed by giving the right answer to the wrong problem” (Kaiser, 1960, p. 134). Once we have appropriately formulated our problem (i.e., thought about it), we must decide what to do about this problem (i.e., act on it). In acting (or abstaining from action), we may encounter a number of errors, to which we now turn.