CHALLENGES IN MIXED METHODS RESEARCH
Despite the many benefits of mixed methods, challenges arise in using and integrating multiple analytic approaches. One of the challenges in using both quantitative and qualitative approaches in designing and testing interventions is to have sufficient time and resources allocated to ensure sufficient rigor for both approaches. Though a qualitative approach using participant interviews and observations may yield detailed information, sampling for qualitative approaches typically involves a small number of people (often <50). Such sample sizes typically preclude the use of statistical testing (e.g., of differences in participant characteristics according to themes that emerged in qualitative interviews; Wittink et al., 2006). The key criterion is to ensure representativeness of the main themes, perceptions, and insights (as we continue to talk to people, do new themes emerge or have we reached “saturation” in that no new themes emerge?), as opposed to estimating a population parameter (which may depend on random selection of a sample large enough to provide statistical power). Another challenge is the initiation of collaboration with investigators who are used to restricting their data collection strategies to quantitative methods to adequately allocate resources to collect and analyze qualitative data from observations, narratives, and visual data (George, 2011). In analysis of mixed data, findings from one strand may be contradictory or discordant with the other strand. Although this may be a challenge or viewed as a weakness, such discrepancies can lead to new insights about the processes or measurements we might otherwise not question.
Fostering transdisciplinary research teams for mixed methods behavioral health research poses a pressing challenge (Kessel, Rosenfeld, & Anderson, 2008). In order to maximize the use of mixed methods in RCTs, principal investigators of clinical trials may benefit from including researchers and staff who are trained in qualitative and mixed method approaches, beginning at the study design stage (Robins et al., 2008). Robins and colleagues (2008) have emphasized that “mixed” teams benefit from time set aside for problem solving and discussions about study design, underlying epistemological assumptions, and interpretations.