STEP 5: Define Behavioral Impact or Health Outcome Rates (Output)

The implementation of the next step involves the documentation of impact or outcome rates of change (effect size range) attributable to the HP-DP program. These data are derived from two sources. An evaluation team needs to conduct a meta-evaluation (ME), or use data and results produced by a prospective RCT for the same problem and comparable population. These steps provide an estimate or document specific behavioral impact and/or health outcome rates produced by an existing versus new HP-DP program. In examining these output data, the methodological issues related to internal validity defined in Chapters 1 through 5 must be addressed. Valid evidence needs to be presented to confirm that the significant changes and E versus C group differences in observed impact or outcome rates are attributable to the HP-DP program. Cost analyses should not be performed unless a methodologically rigorous outcome evaluation has been conducted to produce results with high internal validity.

Estimation of Behavioral Impact and/or Health Outcome

Rates: A Meta-Evaluation

An economic evaluation needs to define what level of effect is a reasonable (defensible) estimate of significant change over time. In some cases, measurement may be complicated, for example, measurement of the quality of adjusted life years (QUALY) attributable to an intervention. However, almost all HP-DP programs, if effective, produce behavioral impact or health status outcomes over a one- to three-year period of successful implementation, for example, high blood pressure, diabetes, or asthma control, reduced high blood cholesterol levels, or behavioral effects such as changes in smoking status or weight. All of these rates can be measured in very valid and reliable ways.

An evaluation needs to conduct a meta-valuation (ME) and a meta-analysis (MA) to define the range (low-medium-high) of intervention effects. For example, as noted in Table 6.7 (Case Study 3), an ME by Windsor (2010) confirmed that the range of cessation rates reported for pregnant smokers (E Groups) exposed to a “best practice” Smoking Cessation and Reduction In Pregnancy Treatment (SCRIPT) program was 12%-27%. This level of impact can be compared to a 0%-11% rates from the usual counseling (C group) rate. The range behavior changes reported in the ME was 4.0%-15.7%: the average E minus C difference was 8.5%.

All eight SCRIPT evaluations met ME criteria for high internal validity. Each evaluation measured behavior change, smoking status, and levels of tobacco exposure at baseline and end of pregnancy by self-report and an independent biochemical test. All evaluations had sufficient sample size and confirmed exposure to treatment or control intervention procedures. Almost all evaluations were conducted among patients in Medicaid-supported prenatal care. These data and methods are essential to estimate the level of effect and costs associated with producing each percent of change for a problem and population for a CEA and CBA. An example of the potential effectiveness and CBA of dissemination derived from an ME of a “best practice” intervention, the SCRIPT methods, for the annual US cohort of pregnant smokers (N = app. 800,000) is presented in Case Study 3 of this chapter.

< Prev   CONTENTS   Source   Next >