STATISTICS IS NOT A SUBSTITUTE FOR SOLID EXPERIMENTAL METHODOLOGY AND DESIGN
JYOTI SAVLA AND DAVID L. LOEWENSTEIN
Statistics are no substitute for judgment.
Scientific research is a process of arriving at a dependable solution to a problem through planned and systematic collection, analysis, and interpretation of data. The study design and data analysis are thus a fundamental aspect of all research studies and especially behavioral intervention research in that they provide investigators the means by which to determine whether the obtained results show reliable differences between one or more treatment or control groups, or are merely obtained as a matter of chance. In many ways, however, the design of a research study is more important than the analysis as no amount of sophisticated data analytic approaches will be able to compensate for the lack of methodological rigor in study design and measurement. Therefore, it is always imperative to obtain input on the study design when evaluating a behavioral intervention research and before a study commences. Consideration of research design is also important because the design of a study will govern how the data are to be analyzed.
In this chapter, we describe a few key issues to consider when designing a research study to evaluate a behavioral intervention. Through examples, we show how research design and analytical techniques are intrinsically tied such that a good study design will lend itself to better analytic techniques and, therefore, yield a better understanding of the phenomenon under study. Most of the issues discussed are relevant to an evaluation of an intervention at any stage of its development along the pipeline, but may be particularly relevant to Phase III efficacy and Phase IV effectiveness studies in which an intervention is compared to a comparison group.