Statistical Power

Economic evaluations conducted alongside clinical trials often suffer from inadequate statistical power. Trials are generally powered to detect a significant difference in the main efficacy or effectiveness outcome, not powered to detect a significant difference in cost. Even if the trials were powered to detect a statistically significant difference in costs, this may not translate into what a decision maker considers a meaningful difference in costs. While a trial could be powered on the basis of cost aims, there are practical challenges to doing so. The first challenge is that sample size requirements for cost aims tend to be much larger than efficacy/effectiveness aims. Reasons for this include a high-level of variation typical to health care costs owing to some participants being high utilizers of care with others being low utilizers of care. This manifests as wide standard deviations in cost measures, therefore making it harder to detect statistically significant differences between groups. In addition, the magnitude of cost differences between study groups may be small, particularly if the behavioral intervention being researched is not designed to specifically affect health care use such as hospital care, outpatient care, and/or medications prescribed. These scientific issues essentially translate to very large sample size requirements for cost studies. Obviously, sample size requirements in the thousands would require much larger study budgets than what is typically available for behavioral research, and may not be possible if the recruitment pool is limited.

To address sample size considerations, it is advisable, if possible, to power the study on the basis of the cost measure. The method used to determine sample size depends on the type of economic evaluation. One approach is to complete traditional power calculations for the cost measure, and then, if a cost effectiveness analysis, conduct a separate power calculation for the effectiveness measure (Gafni, Walter, Birch, & Sendi, 2008). Glick and colleagues as well as the International Society for Pharmacoeconomics and Outcomes Research provide useful guidance on this subject (Glick, 2011; Ramsey et al., 2005).

In many cases, it will not be feasible to power the study on the basis of cost measures. In this situation, an economic model can be developed to estimate costs and/or cost-effectiveness. In the model, initial assumptions, sometimes referred to as the “base case”, can be obtained from the study. Then sensitivity analyses (previously discussed in this chapter) can be performed to test the impact of modifying base case assumptions in accordance with the ranges expected in real-world settings. These ranges can be informed by available literature, external data, variation observed in the trial, or expert opinion.

< Prev   CONTENTS   Source   Next >