Net Loss Statistics, Expected Net Loss Curves and the Expected Net Loss Frontier

The net loss statistic for any given strategy i from a range of multiple strategies (i = in any given replicate is defined following Eckermann et al. (2008) as

This provides the required flexible comparator, appropriately varying the net benefit maximising strategy to compare with across given costs and effects of strate-

Fig. 8.5 Expected net loss curves and the expected net loss frontier comparing six GERD strategies (Source: Eckermann and Willan (2011))

gies in each replicate and the range of threshold values for effects (A) considered, while being consistent across strategies, retaining the additive separability properties of INB (Stinnett and Paltiel 1997).

Where a strategy is the net benefit maximising strategy in any given replicate at a given threshold value, its NL is 0; otherwise, NL is positive and represents the extent of loss in NB relative to the optimal strategy. Average NL across replicates for each strategy at any given threshold value represents their expected net loss for that threshold value. Mapping these across potential threshold values forms expected net loss curves for each strategy (Fig. 8.5) with the expected net loss frontier as ENL curves lower bound identifying optimal strategies across threshold values.

For GERD strategies ENL curves and their lower bound the ENL frontier show that: strategy C minimises ENL (or maximises ENB) from 0 up to 10.26 per week of GERD avoided; strategy A from 10.26 up to 35.02; strategy E from 35.02 to 265.79 and; strategy B for 265.79 per week of GERD avoided or higher.

Analogous to flexible axes on the C-DU plane, the NL statistic for each replicate and ENL curves for strategies as NL averaged across replicates at any given threshold value provides a flexible comparator systematically ensuring that each strategy in each replicate is compared with the net benefit maximising strategy at any given threshold value. Importantly while appropriately flexible, the consistent comparison with a net benefit maximising strategy in every replicate for any given threshold value also simultaneously provides a consistent benchmark to compare all strategies against within and across replicates. Hence, in comparing across multiple strategies, NL statistics in each replicate and ENL curves summarising evidence across replicates provide the appropriately flexible while consistent comparator. The net benefit maximising strategy in any given replicate as the compara?tor for NL at any given threshold value overcomes fixed comparator problems of INB statistics. Importantly, this implies the distance between ENL curves at any given threshold value represents the difference in ENB, noting maximising ENB is the same as minimising ENL.

ENL curves are no more difficult to construct than CEA curves. Both identify the strategy maximising net benefit at a given threshold value in each replicate, and then for each strategy, ENL curves calculate expected loss across replicates relative to the NB maximising strategy in each replicate, while CEA curves calculate the proportion of replicates for each strategy compared that they maximise NB. However, unlike CEA curves, ENL curves and their lower bound across threshold values, the ENL frontier both highlight the strategy minimising ENL or equivalently maximising ENB across threshold value and show differences in ENL between strategies at any threshold value, and hence also explain why. Consequently, ENL curves and frontiers clearly and directly inform the primary concern of societal decision makers under the Arrow-Lind theorem. Whether risk neutral or somewhat risk averse they require being informed of differences in expected net benefit or equivalently differences in expected net loss across potential threshold values.

The expected net loss frontier identifies at any given threshold value which strategy minimises expected net loss or equivalently maximises expected net benefit, with distances between ENL curves of strategies represent differences in expected net loss or expected net benefit. Formally this result arises from the net benefit correspondence theorem (NBCT) which shows a one to one correspondence between maximising net benefit and minimising net loss (Eckermann 2004; Eckermann et al. 2008; Eckermann and Coelli 2013, McCaffrey et al. 2015). This result arises in the case of multiple strategy comparison in this chapter, but also more generally, including multiple effect domain comparison (Chap. 10) and comparing efficiency across multiple providers in practice consistent with net benefit maximising quality of care (Chap. 9). Importantly, and as emphasised in Chap. 9 particularly, this result is robust provided NBCT coverage and comparability conditions are satisfied. Comparability and coverage conditions are implicitly or naturally satisfied, where, as in the GERD example, health technology comparisons are based on appropriately randomised control trial evidence in informing relative treatment effects, cover the range of potentially optimal strategies, and cost and effects are compared over common adequate time periods. However, comparability and coverage conditions by necessity need to become more explicit with comparisons in practice such as those required in Chap. 9 across hospitals. Such practice comparisons in order to satisfy coverage and comparability conditions need to standardise for differences in patient risk factors at admission and either model or data link to cover effects and costs over an adequate common time period (e.g. 1 year from date of admission).

In comparing multiple strategies, comparison of relative expected net loss, or equivalently under the NBCT expected net benefit across strategies, is all risk- neutral societal decision making requires in interpreting cost effectiveness evidence. Somewhat risk-averse societal decision making supports the strategy with

Fig. 8.6 Multiple strategy CEA curves for GERD strategies

lowest ENL (highest ENB), except for discrete threshold regions where trade-offs between expected value and probability might need to be considered. That is, discrete threshold regions where with lack of symmetry for net benefit distributions between strategies another strategy might have higher probability of maximising net benefit in comparison with the strategy maximising expected net benefit.

ENL curves and the ENL frontier directly inform which strategy is minimizing ENL (maximize ENB) relative to other strategies at any threshold value. Over threshold regions where strategies vie for minimising ENL (maximising ENB), the incremental P(max NB) can be informed by CEA curves over those regions. However, for multiple strategy comparisons this should be restricted to bilateral CEA curves between potentially optimal strategies over such discreet regions given incremental probabilities between these potentially optimal strategies are usually confounded by other strategies with multilateral CEA curves.

For example, consider the case of GERD and comparison between strategies B and E around $265.79 per week of GERD avoided (Fig. 8.5). Below $265.79 E has a lower ENL (higher ENB) and above which B has a lower ENL (higher ENB). Multiple strategy CEA curves across the 6 GERD strategies (Fig. 8.6) show the probability that each strategy maximizes NB in comparison of 6 strategies across the 1000 replicates. These multiple strategy CEA curves suggest that strategy E has higher probability of maximizing NB than B up to $272.56 per week of GERD avoided. Combined with ENL curves, this might suggest a trade-off between higher incremental ENB for B and higher probability of maximizing NB with E over the region from $265.79 to $272.56 per week of GERD avoided, a trade-off region appearing to be $6.77 in size.

Fig. 8.7 GERD trade-off region between E having higher probability of maximizing NB than B and lower ENB (higher ENL) (Source: Eckermann and Willan (2011))

However, in comparing potentially optimal strategies B and E, note that the incremental probability of maximizing NB between B and E in Fig. 8.6 in informing such a trade-off for somewhat risk-averse societal decision making is confounded by strategy F over this region. If we remove the confounding impact of F (or more generally other strategies) by restricting CEA curves to bilateral comparison between B and E, then the true picture of the incremental probability of interest between B and E is revealed (Fig. 8.7) with E only having a higher P(max NB) than B in the localized region from $265.79 up to $266.95 per week of GERD avoided (Eckermann and Willan 2011).

Hence, in comparison between potentially optimal strategies, the trade-off region where B has higher ENB and E has higher P(Max NB) is restricted to a discrete region of size $1.16, from $265.79 to $266.95. That is, less than 17% of the $6.77 region suggested with the relationship between B and E confounded by F suggested by multiple strategy CEA curves (Fig. 8.6). Further, in interpreting this already smaller trade-off region, it should be noted that as societal decision making is only somewhat risk averse under the Arrow-Lind theorem, the localized threshold region over which preferences differ from those based on minimising expected net loss is usually expected to be considerably smaller still. For example, consider trade-offs in the region between $265.79 and $266.95 per week of GERD avoided (Fig. 8.7) between strategy B with higher ENB and E with higher probability of maximising net benefit. For somewhat risk-averse decision making under the Arrow-Lind theorem, even at the greatest difference in probability of E maximising net benefit (of 0.016 or 50.8% vs. 49.2% chance of maximising net benefit, between $265.79 and $266.25), a point of indifference with ENB might hypothetically with somewhat risk averse preferences arise at say $0.16 higher ENB for B. Such somewhat risk-averse preferences would imply that only between $265.79

and $265.90 of the trade-off region would E be preferred and diverge from that with risk neutrality.

While the actual discrete region where E might be preferred is an empirical issue where preferences for societal decision making for investment decisions in health care are somewhat risk averse under the Arrow-Lind theorem, the value of higher ENB can be expected to be greater than the value of incremental probability over much of the already discrete trade-off regions. Hence, it should be clear that the impact of societal decision making preferences differing from risk neutrality under the Arrow-Lind theorem is very much at the margins. This is both as ordering of strategies between probabilities and expected values is at the margins and result in very localized threshold trade-off regions, and the impact of those trade-off regions on decision preference will be significantly further diminished by the limited extent of risk aversion. That is, where expected value still largely predominates over probabilities given risk spreading over many decisions and large public health populations impacted.

Finally, an argument could also be made that differences between expected value and probability for any two potentially optimal strategies compared over discrete threshold value ranges may result from random noise of methods employed in estimating INB rather than necessarily being real. That is, if methods were employed where INB distributions between any two strategies were symmetric, then trade-off regions between maximising expected values and considering probability (risk) tradeoffs would disappear altogether. Regardless of where one stands on this argument, it should be clear that societal decision making is predominantly interested in differences in ENB or equivalently ENL in informing investment and reimbursement decisions across multiple strategies based on current evidence.

The bottom line is that ENL curves and the ENL frontier directly inform risk- neutral societal decision making of which strategy is maximising ENB at any threshold region and why.

For somewhat risk-averse decision making, the ENL frontier can be combined with trade-offs over discrete threshold regions where they arise between the strategy maximising ENB and strategies with higher probability of maximising net benefit, informed by relevant bilateral CEA curves to avoid confounding inherent in multiple strategy CEA curves. ENL and relevant bilateral CEA curve evidence is combined in Table 8.2 for GERD strategies.

Table 8.2 GERD advice for somewhat risk-averse societal decision making

^aTrade-offs informed by ENL frontier and bilateral CEA curve to prevent confounding

Further, advantages arise from the ENL frontier in informing societal decision making as the ENL frontier also provides a missing link between research and reimbursement decisions. The ENL frontier simultaneously identifies at any threshold value the strategy maximising ENB (on lowest ENL curve and hence ENL frontier) but also the current per-patient EVPI from adopting that strategy.