Procedures that integrate qualitative and quantitative methods to enhance inference quality

A number of reviews of the ways that mixed methods has been used in disciplines in the business and management fields offer a good starting point to locate examples of mixed methods research, including those that integrated different methods or sources of data in ways that advanced analytical and theoretical insight (e.g., Grafton et al., 2011; Cameron et ah, 2015; Molina-Azorin and Lopez-Gamero, 2016; Gibson, 2017).

A wide range of analytical procedures can be used to integrate qualitative and quantitative methods during analysis for purposes of extending the analytical depth or breadth of the findings. These almost always involve the type of dialogic exchange that occurs when findings from the analysis of one source of data generate ideas about ways to pursue analysis with another source of data. Analytic density can be achieved by strategies that integrate findings from different methods in ways that create the opportunity to discern patterns, detect relationships between constructs, identify conditions that influence the outcome, and that recognise the multidimensionality of core constructs.

Table 10.2 (see p. 183) extracts data from a diverse cluster of empirical articles from business and management that integrated qualitative and quantitative data during analysis. It only includes examples where meaningful integration occurred during data collection and analysis. The table

Table 10.2 Examples of procedures that advance inference quality and their contribution in mixed method research in management research

Innovative

Procedure

Principal Contribution of the Procedure

Examples

Investigate group differences

Explain non-significant or contradictory results.

Kaplan and Duchon (1988)

Quantify qualitative data

Reveal multi-dimensionality of a construct.

Santiago-Brown et al. (2015); Hamann et al. (2017)

Pursue contradictions with additional analysis

Expand an extant theoretical model by adding constructs to it.

Barley et al. (2011); Gardner (2012);

Kaplan and Duchon (1988);

Sutton and Rafaeli (1988)

Elaborate conditions under which a social process occurs and doesn't occur.

Bezrukova et al. (2009)

Case-based analysis

Test relationships observed in cases.

Sharma and Vredenburg (1998)

Suggest and/or confirm relationships between variables.

Hamann et al. (2017); Sharma and Vredenburg (1998)

Process tracing

Generate ideas about causation associated with a process that is time-ordered.

Gardner (2012); Kaplan and Duchon (1988); Vaast and Levina (2015)

Visual display

Generate hypothesis or theoretical propositions for further analysis.

Gardner (2012); Hamann et al. (2017)

pinpoints the analytical procedure used, the principal contribution of the procedure to analytical insight, and lists a few publications that utilise the procedure.

The transparency afforded by labelling research as ‘mixed methods’ varied across the entries listed in Table 10.2. Some describe their research as combining qualitative and quantitative methods. Some label their approach as multi-method. Very few facilitated the act of locating examples of research about sustainability that used mixed methods by explicitly including these words in the title or abstract.

The examples highlighted in Table 10.2 integrated methods for purposes of elaboration or expansion of conceptual understanding that is achieved when data from one source informs data collection or analysis in another. The fact that most entries appear more than once in the table reflects that integration often serves multiple purposes in studies using an iterative design.

Two Case Exemplars that Integrated Methods to Advance Inference Quality

In this section, we put under the microscope two examples from the management literature that expand conventional approaches to mixed methods in several ways. First, they are unusual in that they illustrate mixed method analytical strategies used in the service of what Fielding (2008, 2012) referred to as analytical density. Secondly, they link an iterative design with that outcome. There is an on-going “back and forth” between different methods as new interpretations emerge and are weighed against different sources of data in an iterative design. A third critical distinction highlights the contribution of dissonance to analytical insight. The first two cases pursued dissonance between the different methods with additional, probably unplanned, rounds of data collection and/or analysis.

Table 10.3 (see p. 185) contrasts key elements of the three critical cases, including one that is designated as a contrasting critical case. It identifies the design, integrative strategy used by each, and identifies the contribution of the integrative analytical procedure.

The first two cases used a hybrid mixed method design with an iterative phase. A hybrid design is an advanced design that incorporates both concurrent and sequential phases (Schoonenboom and Johnson, 2017), but is not always iterative. The contrasting case used one of the basic designs where the quantitative and qualitative strands are compartmentalised. Integration for purposes of explanatory insight is not a priority in this study.

I use the same organizational template to summarise information about each of the three case exemplars. The section about each article is organised in three parts: purpose of the research, a description of how integration occurred, and a summary of the analytical contribution of integration.

Case 1: Kaplan and Duchon (1988): Reconciling conflicting findings by re-analyzing the quantitative data with new constructs introduced during the analysis of qualitative data

Purpose: It is quite a common experience, particularly in research gauging the outcomes of an intervention, to find that the quantitative results and the qualitative findings point to different conclusions. This was the experience of a pair of collaborators reporting on a pilot study they described as combining qualitative and quantitative methods. This study has an explanatory, hypothesis testing drive. It was designed to explore the relationship between orientation to work and what happens when a new management information system is installed in commercial laboratory settings.

Extending the value-added 185

Table 10.3 Integrative procedure, generic label, and contribution to analytical insight for two critical cases in method

Example

Integrative Procedure

Contribution of the Procedure to Analytical Insight

Kaplan and Duchon (1988)

Quantitative data reanalysed with constructs identified in the qualitative analysis.

Confirmed grounded theory model that explained differences between groups.

Gardner (2012)

A case-based mapping activity.

Illustrate how paradoxical results play out in interaction patterns during meetings.

Integrative procedures: Results from an independent analysis of the qualitative and quantitative data conducted by two different researchers were contradictory. The initial analysis of survey data provided no statistical support for the findings from the qualitative data analysis that pointed to strong differences in the work orientation of laboratory team members in different settings. Following the concurrent analysis of the qualitative and quantitative data, the member of team who described her expertise as qualitative, developed a grounded theory model that foregrounded work orientation as a mechanism underlying the differences between attitudes toward the newly implemented management information. In the final phase of analysis, the quantitative survey data were re-analysed with the addition of variables constructed to represent the constructs or themes emerging from the grounded theory.

Contribution of integration to analytical insight: Results of the integration of the qualitative data both confirmed and elaborated findings. The reanalysis of the quantitative data confirmed the grounded theory model developed following the completion of the qualitative phase. It elaborated the conceptual framework by providing statistical support to document the intermediary role of work orientation in explaining attitudes about the implementation of new management software.

Case 2: Gardner (2012): Leveraging a mapping activity and case-based analysis to explain paradoxical findings

An example what the author refers to as multi-method research by Gardner (2012) demonstrates ways that a “back and forth” exchange between qualitative and quantitative strands can generate new analytical insight. Most specifically for our purposes here, we examine it to consider the way that a visualization like a diagram that maps interactions can be used to explore paradoxical findings. The description Gardner provides of the analytical procedures could be helpful to others looking for ways to analyse observational data of team meetings.

Purpose: Gardner set out with the purpose that applies to almost any type of workplace team. She sought to explore the link between team knowledge and performance by using what began as an explanatory sequential design but shifted to a hybrid design. Gardner’s objective was to unravel the puzzling finding that teams often default to the person, probably more senior, with non- specialised knowledge in the very setting where high performance pressures demand the expertise provided by a team member with specialised knowledge. She added a third, qualitative phase to the project where she used process tracing. Process theory deals with events and the processes that connect them (Maxwell et al., 2015). Gardner drew process diagrams to map the sequence of actions and interactions she observed in team meetings.

Integrative procedures: Gardner (2012) used a type of case-based analysis with a process-oriented mapping activity. Not to be confused with a case study that serves a summative purpose in a final report, case-based analysis is a mixed method analytical strategy that integrates qualitative and quantitative data in a narrative form or through a visualization (Bazeley, 2018). According to Pat Bazeley, an Australian and prolific textbook writer, ‘Each case holds data from different sources and different types together, thus cases provide the lynchpin for integration of data’ (2018, p. 26).

The maps developed by Gardner (2012) served the same role an analytic memo might serve in a project with a grounded theory component. She recorded each incident of knowledge use and used a symbol system to distinguish between the language used by the general professional experts and domain-specific experts. Each map was annotated to provide data to support her interpretation. In the two examples of maps included in the article, time pressures may have explained why the views of the more experienced, but less technically proficient, member prevailed.

Contribution of integration to analytical insight: Findings from the crosscase comparison of the process maps are not elaborated. Gardner makes that argument that they supported generalizability by confirming findings from the analysis in earlier phases of the quantitative data. There is a lot more this author could have learned by further exploring the data in the process maps.

The third and final critical case by Mahmood et al. (2018) is a contrasting case. It addresses the issue of corporate governance structure and sustainability reporting. It further illustrates just how closely the approach taken to integrating research methods (and often team dynamics) influences the potential to gain analytical depth in a mixed method study.

Contrasting case example with a non-iterative design. Case 3 Mahmood et al. (2018): Research in sustainability with a non-iterative design

A mixed methods study appearing in the journal, Sustainability, by Mahmood et al. (2018) further magnifies the distinctions between iterative designs and more basics ones. This is the only case where the authors show familiarity with a slice of foundational literature about mixed methods. It’s the only case to offer transparency by using the expression “mixed methods” in the title, by explicitly labelling the research design, and by using language to explicitly acknowledge how integration occurred.

This example has the advantage of using a widely recognised, core mixed method design. It aligns an explanatory sequential design with a triangulation purpose. The drive is quantitative and the rationale for integration is triangulation or validation. The purpose, integrative strategy, and outcomes achieved differs markedly for this research than for the two cases previously presented.

Purpose: The aim of the research was to explore the relationship between elements of corporate governance and disclosure sustainability. Results supported research documenting the link between board size, but not gender diversity, and initiatives to report sustainability initiatives.

Integrative procedures: In the “materials and methods” section the authors describe a study with a distinct quantitative phase that was followed by a second, qualitative phase. Integration in this case occurred at a single point and in a procedural, rather than conceptual way. That was at the juncture between the two phases when results from the quantitative analysis were used design the qualitative instrument and to guide selection of participants.

Contribution of integration to analytical insight: The qualitative phase in this study is so secondary as to make it unlikely that it could be published separately. Only a very small number of board members were interviewed in order to validate the quantitative results. The qualitative data were used to illustrate the quantitative findings, but there was no separate emergent analysis. Contradictions between the qualitative and quantitative findings from the different methods were acknowledged but not pursued.

 
Source
< Prev   CONTENTS   Source   Next >