Combining Study Findings by Using Multiple Literature Review Techniques and Meta-Analysis: A Mixed-Methods Approach


A mixed-methods research approach is defined as a procedure for using a combination of quantitative and qualitative methods for the collection and analysis of data in the same research study (Creswell and Clark, 2011). However, a multi-method approach is differentiated from a mixed-methods approach in not requiring both quantitative and qualitative methods. Instead “multi-method” refers to the use of multiple methods that are either qualitative or quantitative. A mixed-methods approach is justified when the interaction between quantitative and qualitative methods provides better analytical possibilities than either method alone. Such an approach becomes relevant when researchers intend to perform comparative analyses simultaneously and develop aspects of the study in comprehensive and in-depth terms. However, although mixed methods have gained visibility in recent years, it is necessary that care is taken to avoid methodological and design problems, with over-simplification being a common criticism (Cheek, 2015, p. 630). Rigour is required when integrating evidence within qualitative and quantitative modalities, or traversing the boundaries that separate them. This occurs in studies where the strength of a quantitative analysis, the purpose of which is specifically to confirm effects, is combined with the deep explanatory descriptions obtained from qualitative analyses (Castro et al., 2010).

In this chapter, meta-analysis (MA) is regarded as a mixed method to combine the qualitative component of literature review (LR) to establish a context in which to use quantitative analysis to combine findings, expressed as comparable quantities, across multiple studies. The LR provides a conceptual basis for ensuring that all studies included are comparable, as are the quantities (Pare et al., 2015). In addition, the use of qualitative information might modify the contribution of each study to the analysis. Studies may be re-weighted by sample size or other measures of quality, such as the extent of important sources of bias, for instance, the use of expert assessments. Important aims of M A include: accumulating knowledge; highlighting consistency amongst studies; detecting potential publication bias; and revealing points of consensus or dissent across such studies. In this way, the conclusions produced by MA can be stronger than the findings of any individual study, owing to: the increased amount of evidence, the ability to account for variability between studies as well as within studies, and the cumulative view of outcomes (Cooper et al., 2019). These statistical benefits can be boosted when LR is “mixed in” as a qualitative research method within the MA, including benefits such as: characterisation of the sampling frame, quantities of interest, and of their quality. In this sense, the purpose of this study was to explore the process of mixing LR methods into MA from two fundamental perspectives. First, it is essential to give a theoretical description of this process. It is not always easy or possible to mix both approaches. Therefore, it is important to explore how mixing methods affects the details of implementation for both the LR process and the statistical modelling within a MA. Second, these issues are illustrated in this chapter by using two scenarios in different fields (natural sciences versus business/ IT) to demonstrate how this process could be applied, whilst identifying useful practices, difficulties, and challenges.

The aim of this study was to provide a fundamental guide for researchers who seek to integrate LR methodologies into a MA using a mixed-methods approach. This will help a researcher to understand the approach, learn how to apply this research methodology, and assess its robustness, challenges, and difficulties during the implementation process.

Quality appraisal using AMSTAR2

Essentially MA begins with LR, potentially modified by experts, to locate relevant studies and extract relevant information about each study, which is then collated across studies using statistical analysis. Criteria are available not only to help guide researchers in good practices for MA, but also to list minimal key requirements of MA. Many criteria are based on the assumption that a systematic LR has been undertaken. AMSTAR2 (Shea et al., 2017) is an update to the original AMSTAR, an instrument widely used to appraise the quality of published, systematic reviews for MA (Farrah et al., 2019). The updated AMSTAR2 criteria can be used to assess both randomised studies (prevalent in medical research) as well as observational and other non-randomised studies (more common in social science). Seven critical domains that broadly correspond to stages of MA are addressed by AMSTAR2 in different items, respectively, as follows:

  • 1 Protocol is stipulated before commencing the literature search (AMSTAR2 Item 2).
  • 2 The literature search is adequate (Item 4).
  • 3 Justification is provided for excluding particular studies (Items 3, 5, 7).
  • 4 Quantities extracted for MA are comparable and adjusted for biases in individual studies (Items 6, 9, 10, 14).
  • 5 Statistical methods of MA are appropriate (Item 11).
  • 6 Interpreting the results accounts for risks of bias (Items 12, 13, 16).
  • 7 Presence and likely impact of publication bias is assessed (Item 15).
< Prev   CONTENTS   Source   Next >