A key consideration in the selection of a sample for an intervention study at any stage of the pipeline is the extent to which the sample is representative of the population for whom the intervention is intended on relevant characteristics. For example, within caregiver research, it is generally recognized that a “one-size-fits-all” approach is not efficacious with respect to the design of interventions, as caregivers vary along a number of dimensions: ethnicity/culture, gender, and age; caregiver experience, needs, and well-being; and caregiving demands, roles, and responsibilities. In the Resources for Enhancing Alzheimer’s Caregiver Health II (REACH II) trial (Belle et al., 2006), the sample of caregivers included three different race/ ethnic groups: White, African American, and Hispanic caregivers, who varied in age, relationship to the care recipient, and years in the caregiving role. This facilitated an analysis of whether the impact of the REACH II intervention varied by geographic, race/ethnic, and other characteristics of caregivers such as relationship to the person with dementia. Answers to these questions are important to help guide further refinements of an intervention and to determine if an intervention benefits some groups over others. Although participants in REACH II represented one of three race/ethnic groups, other subgroups were not represented such as Asian and Haitian caregivers. The exclusion of these groups limits the generalizability of findings from REACH II to these caregiver populations.
The composition of a sample is also important in another respect. The sample must possess the characteristics that the intervention intends to address or modify. Take, for example, a caregiver intervention that is designed to reduce depression and distress. As not all caregivers are depressed or find caregiving distressful, the sample for a study to test the effects of a depression intervention would need to include only a subset of caregivers—namely those who have depressive symptoms. Aligning the characteristics of the sample with the intent of an intervention through the specification of inclusion and exclusion study criteria is critical; otherwise, it would not be possible to adequately demonstrate whether the intervention has an impact or not and for whom.
In essence, a representative sample is one that accurately reflects the members of the population for whom the intervention is targeting; in other words, it is one that has strong external validity with respect to the target population of the intervention (Davern, 2008). Having a sample that is representative enhances the confidence with which the findings from a study can be generalized to the population that is the focus of the intervention. A sample that is not representative leads to bias or sampling error; certain groups may be overrepresented and others may be underrepresented, which impacts on the outcomes of the study. For example, in the Personalized Reminder Information and Social Management (PRISM) trial (Czaja et al., 2014), extensive pilot testing was used to evaluate the usability of the PRISM software before the implementation of an efficacy trial. The target population for the randomized controlled trial (RCT) was older adults (age 65+) who were at “risk for social isolation” and had minimal prior computer/ Internet experience. If the sample for the pilot testing had included middle-aged adults with extensive computer or Internet experience, the findings regarding the software usability would have been biased and have limited generalizability to the target population. This illustrates the importance of carefully considering who the sample should be early on in the intervention development process and assuring representation of that sample through careful construction of study eligibility and ineligibility criteria and recruitment strategies.
Sample bias cannot be totally eliminated; however, it is important to attempt to minimize bias to the extent possible and also to understand the limitations imposed by the sample included in an evaluation of an intervention. Three factors influence the representativeness of a sample: sample size, sample attrition, and sampling method. With respect to sample size, the larger the sample the more likely it is to be representative of the target population and thus the less likely it is to be biased. Sample attrition can also lead to potential bias if those who drop out of a study have common characteristics (e.g., those who are older or who have less skill) as the remaining study participants will no longer be representative of the original sample.
This can lead to overestimation of the intervention effects or misestimating the effects. For example, assume an investigator is interested in evaluating the impact of a stress reduction intervention on the depression of family caregivers and the original sample includes caregivers with varying levels of depression. The results of the study indicate that the study is efficacious and results in a significant decrease in caregiver depression. However, during the course of the study, caregivers with high levels of depression have higher dropout rates. In this case, the impact of the treatment may be overestimated as the sample that received the intervention consists primarily of caregivers who have low levels of depression. Finally, sampling method can also influence bias as there are various ways to choose a sample, and as will be discussed later in this chapter, there are sampling methods that can be used to help ensure that the sample is representative of the target population.
In general, who should be included in a study depends on the objective(s) of the intervention and the research question(s), research design, available resources including budget, and sample availability. It is critical to characterize the target population of interest before a sample can be defined. For example, assume a researcher is interested in determining if an intervention that involves cognitive behavioral therapy delivered via videoconferencing is effective in alleviating symptoms among people with emotional disorders. It would be important to narrow and refine the research question to specify the type of emotional disorder (e.g., depression, bipolar disorder), age range (e.g., adolescents or adults), living arrangement of the participants (community-dwelling or hospitalized patients), and any restrictions with respect to medications or substance abuse. It would also be important to consider the availability and accessibility of a sample in terms of any potential geographic or time constraints in recruiting, enrolling, and retaining the targeted group. In the PRISM study, if the majority of the potential participants lived in an underserved area, Internet access may have been spotty, which would influence the size of the pool from which the sample can be drawn. This may also limit who may be able to enroll in the study and successfully engage in the intervention. The decisions made about sampling have implications for recruitment and retention strategies (Chapter 10).
Another important consideration in sampling is the inclusion of women and minorities. When designing interventions, it is important to consider culture/ ethnicity and gender as these factors can moderate or directly impact on the outcomes of an intervention. Many funding agencies, particularly in the United States, require applicants to clearly state if women and minorities are included in a study. In the United States, the National Institutes of Health (NIH) insists that women and members of minority groups must be included in all NIH-funded research, unless a clear and compelling rationale and justification are otherwise provided that their inclusion would be inappropriate with respect to the health of the subjects or the purpose of the research. This issue is considered in scientific peer review. As stated on the NIH website:
Peer reviewers will also assess the adequacy of plans to include subjects from both
genders, all racial and ethnic groups (and subgroups), and children, as appropriate,
for the scientific goals of the research will be assessed. Plans for the recruitment and
retention of subjects will also be evaluated. (NIH, 2015)
Of course, the nature and scope of diversity of the sample must be based on the research question posed; however, lack of diversity in a proposed sample must be clearly and adequately justified. For example, it would be reasonable to include only women in a study that is evaluating an intervention aimed at alleviating depression in women recently diagnosed with breast cancer, but it would not be reasonable to restrict the sample to White women. The NIH also requires consideration of the inclusion of children (<18 years of age), and a rationale must be provided in grant applications for their inclusion or exclusion. As noted by Kazdin (1999), there is a need to sample broadly, to evaluate the moderating role of sample differences, and to pursue mechanisms through which moderating factors may operate. Sampling broadly along a number of characteristics pertinent to the intervention also enables an intervention to have maximum “reach” to all those who are intended to benefit.