Cost Minimisation and the Absence of Evidence Fallacies
Drummond et al. (1987) suggested cost minimisation analysis as appropriate where there is ‘no statistically significant difference in relative effect’ between trial arms, proposing that only direct cost of interventions is considered in such cases. This approach effectively sets up a partial sequential hypothesis test where the hypothesis is tested that effects are statistically different at a pre-specified level of type I error (typically 5% with a two-sided test) and then:
- (i) If effects are tested as different, consider cost effectiveness, cost benefit and cost utility analysis; or
- (ii) If effects are not tested as different, then cost minimisation analysis is indicated.
For example, consider comparing interventions A and B, where the relative risk for a 12-month survival for A versus B is estimated as 0.50, with 95% CI (0.20,1.20) and the direct cost of A is $1000 less than B (say $19,000 vs. $20,000). Following Drummond (1987), the lack of statistical significance of the trial finding with a 5% two-sided type I error leads to the following inferential trial:
- (i) Infer an equivalent 12-month survival between A and B from a lack of statistical significance against a 5% two-sided type I error.
- (ii) Cost minimisation is justified by 1.
- (iii) A is inferred to weakly dominate B with $1000 lower direct cost and ‘equivalent’ effect from (i).
Briggs and O’Brien (2001) highlight that problems with the inferential trial for cost minimisation analysis as proposed in Drummond (1987) arise by inferring equivalence of effect from a hypothesis test against a probability of type I error in:
- (i) Not allowing for type II error (probability of false negative) or associated notions of powering in testing with difference trials, rather than equivalence trials; and
- (ii) Ignoring expected cost implications that arise from differences in effects (i.e. ignoring cost and effect covariance).
Given (i) and/or (ii) a weak dominance inferred from the inferential trial followed by Drummond (1987), can easily represent a dominated or cost ineffective strategy. Further, (i) and (ii) also imply that incentives are created for manufacturers or more generally those with a vested interest in promoting an intervention where initial evidence points towards such interventions being inferior to collect inadequate evidence. That is, by treating absence of evidence as though it were evidence of absence (Altman and Bland 1995), scant evidence can be collected for likely inferior interventions and then be ignored in the absence of statistically significant differences.
For example, in the case of A versus B, while the direct cost of A may be $1000 cheaper, A is expected to have half the survival rate of B at 12 months, which for a late-stage cancer population might translate to a 10% versus 20% survival rate at one year.
If we directly present this evidence on the incremental cost effectiveness plane only comparing A versus B, then the point estimate of an ICER for A versus B would lie in the south-west quadrant, where a $1000 reduction in costs for A is at the expense of an expected 10% absolute reduction in survival rate. This clearly does not indicate dominance of A relative to B. Indeed if such interventions have expected lower effect while cost saving (in the SW quadrant), they should be compared with the best alternative actions in raising funds for investment, as discussed at length in Chap. 11 following Eckermann (2015). That is, interventions or strategies with incremental expected costs and effects in the SW quadrant should be compared with the best alternative disinvestment action that with the lowest reduction in health expected to arise in saving the health system costs, in order to fund alternative actions, or equivalently the greatest cost saving (funding raised) for any given health loss.
More generally, the absence of evidence being treated as evidence of absence (Altman and Bland 1995; Briggs and O’Brien 2001) is particularly problematic with cost effectiveness analysis where effect inference is based on a type I error as proposed by Drummond (1987), given biases that can result for cost as well as effect estimation and their interaction, in informing cost effectiveness estimates and decision making. That is, effects ignored are usually also expected to have cost implications for the health system where they have associated treatment, such as hospitalisation for various forms or morbidity or treatment of side effects. Hence, an intervention which on the face of a cost minimisation analysis has lower direct cost and ‘equivalent effect’ can easily represent worse effects, but also higher health system costs in appropriately including the cost of treating such effects. What cost minimisation following Drummond (1987) suggests as weak dominance (‘equivalent effect’ and lower direct strategy cost) can therefore easily represent a dominated strategy in reality.
To avoid inferential fallacies that arise with cost minimisation as proposed by Drummond (1987) and more generally clarify cost effectiveness analysis, the focus should be on jointly presenting estimates of incremental costs and effects relative to relevant comparator/s and ideally their joint density under uncertainty (Briggs and O’Brien 2001). Consequently, critiquing cost minimisation analysis leads to the same conclusions for principles and methods required for robust cost effectiveness analysis as critiquing of the box method, following Briggs et al. (2002) in Chap. 2. Costs and effects should always be considered together to avoid partial analysis biases that otherwise arise in consideration of societal decision making in relation to cost effectiveness, or net benefit, analysis.