Measurement of Epistemological Development
This chapter provides an overview of multiple methods and tools that have been used in measuring epistemological development, including both qualitative and quantitative methods.
Qualitative Measures of Epistemological Development
Throughout the past 40 years of research in the cognitive developmental field, primarily, qualitative methods are mostly acknowledged and recommended for providing in-depth, rich details of students’ epistemological understanding. It has been adopted as a more reliable way by which to understand the students’ cognitive developmental stage as portrayed in various former studies, including those of Perry (1970), Belenky et al. (1986), Baxter Magolda (1992). The drawback of using qualitative methods in these prior studies obviously is the intensiveness of one-on-one interviews and the data analysis process. In addition to the intensiveness of data collection and analysis, the nature of qualitative research limits the possibility to perform a large-scale estimation of the cognitive development of a large student population. For each series of studies, I will summarize a brief discussion about their interview process and methods.
The central concern of Perry’s work was students’ own reports of their college experiences as expressed in the students’ own languages. Therefore, phenomenology was used to guide their interview process. To avoid biasing the students’ thought structure through the structure of the interview protocol, interviews were conducted in as open-ended a way as possible. The interview was started with the simple question: “What stands out for you about the year?” (Perry 1970, p. 19) and refused to provide further specific questions. This attempt aimed at encouraging the participants to express varied ideas and to make sense of their personal experiences in their own way. This attempt and the genuine interest in uncovering the students’ own experiences through their own language were compensated by the students’ taking the lead in expressing their ideas. Perry acknowledged that this procedure caused the beginning moments to be “socially awkward.” However, as Perry stated, this procedure was crucial to the research in that the primary focus of the research was the participants’ own way of making sense of his or her experience. As a result, this method also proved to be effective in helping the many students to express their personal ideas.
For Belenky et al.’s research (1986), they organized their interview protocol under different sections. As I noted under the discussion about their theoretical framework, there was potentially a serious concern in their methodology (Hofer and Pintrich 1997). In the interview protocol, a section on “Gender” and a section on “Relationships” were placed before sections on “Real Life Moral Dilemma,” “Education,” and “Ways of Knowing.” Considering their main findings focused on women’s self-knowledge, the relationship between self-knowledge and knowing, inner and outer voices, and the connected characteristics of women’s way of knowing, it is hard to tell the degree to which the interview questions have affected these main findings.
For Baxter Magolda’s studies (1992), she retained the interview method. In addition, she classified interview questions into six content areas based on previous work by Perry (1970), Kurfiss (1977). These six content areas included (1) the role of learners, (2) role of instructors, (3) role of peers, (4) perception of the evaluation of their work, (5) nature of knowledge, and (6) educational decision-making. Within each content area, she retained an un-structured interview procedure, i.e., where interview questions were asked to introduce the topic but not frame the response. By segregating different content areas, she was then able to present a clearer picture of students’ views in each of the researched areas. Considering the advantage of this interview structure, this interview structure will be adopted as part of the methodological design of this work (see Chap. 5).
Reflective Judgment Interviews (RJIs) were developed by King and Kitchener (1994). The Reflective Judgment Interviews have five standard problems on questions like the objectivity of news or food safety. They also have additional discipline-based questions (Psychology, Chemistry, and Business). Based on the interviewee’s understanding and justifications of the claims, raters will rate the interviewee as responsive with regards to two aspects: (1) the nature of knowledge and (2) the nature of justification. As discussed under the theoretical framework section, it has been widely used by a large number of participants (King and Kitchener 2002). However, this process may lead to concerns with regards to disciplinary issues. An intensive study in one discipline may potentially affect their reflective thinking in that discipline. Therefore, this measure may produce artifacts on measures across disciplines.
