I: Critically designing and evaluating diary studies
Using diaries in mixed methods designs: Lessons from a cross-institutional research project on doctoral students’ social transition experiences
Jenna Mittelmeier, Bart Rienties, Kate Yue Zhang and Divya Jindal-Snape
In higher education research, as well as across the wider social sciences, there is increased awareness that combining multiple research methods enables researchers to better understand the complex and often non-linear phenomena that occur in today’s societies (Creswell & Creswell, 2018; Creswell & Plano Clark, 2017; Greene, Caracelli & Graham, 1989; Johnson & Onwuegbuzie, 2004). In this chapter, we consider solicited diaries as one possible method to incorporate within a mixed methods higher education research design. We start by defining mixed methods research broadly and outlining various approaches, purposes and key considerations for its use. The second half of this chapter highlights the benefits and challenges of mixed methods diary designs through reflection on our own research examining doctoral students’ social community building experiences. The chapter closes with six key considerations for higher education researchers for developing rigorous mixed methods diary studies.
Approaches and considerations for mixed methods research
While there is debate about how to label and define mixed methods research, we define it in this chapter in line with Bazeley (2017, p. 7): ‘Any research that involves multiple sources and types of data and/or multiple approaches of those data, in which integration of data and analyses occurs before drawing final conclusions about the final topic of the investigation.’ Some mixed methods researchers argue that mixed methods must involve both quantitative and qualitative elements (see, for example: Johnson, Onwuegbuzie & Turner, 2007), but there is increasing recognition that this strict dichotomy between research paradigms is perhaps superficial (Bazeley, 2017).
Mixed methods diary research offers many benefits for approaching complex research topics, including those present in many higher education studies. One consideration is that researchers can draw upon the strengths and counterbalance the weaknesses of chosen methods, as well as more broadly balance quantitative versus qualitative research paradigms (Symonds & Gorard, 2010). One additional strength of mixed methods approaches is that they allow for data triangulation by combining multiple insights into a phenomenon from different perspectives. As described by Cohen, Manion and Morrison (2018, p. 195), ‘triangular techniques in the social sciences attempt to map out, or explain more fully, the richness and complexity of human behaviour by studying it from more than one standpoint.’
Mixed methods designs are inherently flexible and allow researchers to think pragmatically about research approaches, paradigms and methods that best answer their research questions (Johnson & Onwuegbuzie, 2004). In higher education research, the underlying pragmatism of mixed methods research is well suited, given that research topics often combine more subjective perspectives (such as students’ experiences or preferences) with more positivistic approaches (such as measuring students’ learning behaviours or progression routes) (Scoles, Huxham & McArthur, 2014).
Despite this flexibility, there are many considerations researchers must make in designing mixed methods diary' research, including managing the sample across multiple methods, developing complementary' research instruments, deciding on approaches to analysis, and outlining procedures for drawing together findings from multiple sources (Teddlie & Tashakkori, 2009). Plano Clark and Ivankova (2016) also note that considerations must be made related to timing (when and in what order to collect data), priority (the relative importance of the various research strands in light of the research questions), and integration (the strategies used to combine multiple sets of data throughout the research design). Taken together, this means there is no single, ‘right’ way' to design a mixed methods diary study, as approaches are often complex and unique to the aims of the individual study. Indeed, Creswell and Plano Clark (2017) highlight six overall approaches for developing a mixed methods research design, all of which could be incorporated into a solicited diary' study:
- 1 Convergent parallel design: data from multiple methods are collected separately, but at the same time, and then drawn together in the analysis.
- 2 Explanatory’ sequential design: an initial quantitative data collection phase is followed by' qualitative data collection, explaining the initial results of the quantitative analysis.
- 3 Exploratory' sequential design: an initial qualitative data collection phase is followed by quantitative data collection, allowing for an initial exploration phase before bringing the study' to scale.
- 4 Embedded design: data from multiple methods are collected together at the same time in one single research phase.
- 5 Transformative design: a transformative theoretical framework is used to inform all decisions within the mixed methods design.
- 6 Multiphase design: convergent and sequential designs are combined over a larger programme of study.
Ait additional consideration in mixed methods research is the careful design of when, where and how multiple methods are integrated throughout the research process (Bazeley, 2017). Woolley (2009, p. 7) describes integration in mixed methods research as:
the extent that these components [multiple research methods] are explicitly related to each other within a single study and in such a way as to be mutually illuminating, thereby producing findings that are greater than the sum of the parts.
Decisions related to integration are important to mixed methods diary designs, particularly as there are multiple approaches to integrating data in mixed methods research (see Creswell and Plano Clark 2017).