Design #i: One Group Pre-test and Post-test
As a pre-experimental design, Design #1 is the most basic method for program assessment. It should be not used to assess program behavioral impact or health outcome over any extended period of time, for example, > 12 months. In this design, baseline measurements are made (O1), an intervention (X1) is provided, and follow-up observations-measurements (O2) are performed. Attributing an observed significant change that occurred between O1 and O2 to the intervention (X) requires an evaluation to systematically explain how it controlled for Measurement, Selection, and Historical biases. For example, did other historical events, unplanned exposures, or unexpected activities of program participants between O1 and O2 partially or fully explain an observed significant change? The longer the period of time between O1 and O2, the more probable it is that an internal or external historical (H) event, unplanned exposures, or program changes may have influenced participant behavior and affected program results. Measurement or selection biases may also explain any observed changes between O1 and O2.
Design #1 can be very useful, however, in conducting an immediate/ short-term assessment of an existing HP-DP program. A program may decide to assess the immediate impact (1-6 months) of an intervention for a specific problem (elderly falls) or a specific condition (high blood pressure control). The interval between O1 and O2 must be short, and the evaluation planned and successfully implemented so that selection and historical biases are implausible explanations of a significant impact. If the baseline and follow-up measurements are valid and complete and occur prior to and soon after the intervention, for example, a few weeks/months before and after, historical bias may not be a plausible threat to impact results.
A process evaluation is essential to confirm successful implementation of intervention and assessment procedures.
Maximum control over measurement quality and data-collection processes must be asserted to control for this bias, regardless of the size or evaluation purpose. Confirmation of data validity and sample representativeness are essential in Design #1, and all designs. If measurement validity is confirmed and the time period is short, the first threat to internal validity for Design #1 will be selection bias. Did the program document high (100%) O1 and O2 assessment rates (> 90%) for evaluation clients? What is the extent to which evaluation participants are comparable or different from other users at HP-DP program sites? The following is an example of the use of Design #1.
