Quantify the Potential for Improvement
The seventh consideration that needs to occur in the discovery phase is quantifying the potential for improvement in a problem area. Improvements from an intervention may be small, medium, or large and include both subjective and objective outcomes including cost (see Chapters 14 and 15 on measurement and Chapter 18 on cost). Projections of the effect size of an intervention can be derived from theory and existing evidence. Projections will inform sample size considerations in the evaluative phases that test for the outcomes of an intervention (see Chapter 9 on sampling considerations). An expected small impact on an intervention (referred to as “the effect size”) will require the evaluation of the intervention with a large sample; alternately, an expected medium or large impact or effect size will not have the same requirements with respect to sample size. However, sample determination is not only a function of size but also includes other considerations such as sample composition and feasibility with respect to recruitment and attrition.
For the ABLE Program, at the time of its development, there was little empirical evidence as to what the effect for the main outcome, reduction in functional difficulties, might be possible to achieve. Pilot testing to advance the intervention was important in order to demonstrate proof of concept and if change in functioning could be achieved. This is an example of how these prephase considerations overlap with initial pilot testing, and this is referred to in Chapter 2 as “Phase I activity.”
On the basis of pilot studies demonstrating proof of concept of the ABLE approach (Gitlin & Burgh, 1995; Gitlin, Schemm, Landsberg, & Burgh, 1996), it was subsequently hypothesized that a medium effect size could be achieved in reducing functional challenges and improving self-efficacy in managing daily challenges. Power calculations with this and other assumptions led to the conclusion that a Phase III efficacy trial would require a sample size of 319 study participants in order to demonstrate a medium effect of the intervention on functional difficulties when compared to a usual care control group.