Data Collection and Analysis

To check the content validity of the survey, eight experts were invited to rate the extent to which each survey item reflected the following positions of Perry’s theory: Position 3-Multiplicity Subordinate; Position 4a-Multiplicity Correlate; Position 4b-Relativism Subordinate; Position 5-Relativism. These external experts represent a variety of perspectives and experiences with Perry’s theory. According to their self-reports, they have either engaged in research efforts related to Perry’s theory, taught some classes related to Perry’s theory, or attended training sessions (classes, workshops, etc.) related to Perry’s theory. One of the experts self-indicated that he had led numerous workshops on the Perry scheme and had been active in research on the scheme for many years.

Table 5.2 presents a sample item of the Expert Rating Form as presented to content experts. All experts rated each item along Positions 3 to 5 using a Likert scale from one to four (1, not at all; 2, only a little; 3, somewhat, 4, a great deal). An open-ended question was also added to collect any additional comments to the survey items. An information page was provided to the experts with brief definitions of each position within Perry’s theory. The information page and the Expert Rating Form can be found in Appendix A. After the experts rated all of the items, the ratings across content experts for each position of each item were compiled. If the sum of external experts rating for a certain position of an item is higher than 24,

Table 5.2 A sample item on the expert rating form

Directions: Please rate the degree to which the statement reflects the view of Perry positions (P) 3, 4a, 4b, and 5 using the following scale: 1: not at all, 2: only a little, 3: somewhat, 4: a great deal

Survey item

P3-

Multiplicity

subordinate

P4a-

Multiplicity

correlate

P4b-

Relativism

subordinate

P5-Relativism

12 3 4

12 3 4

12 3 4

12 3 4

E.g., Everyone has a right to his/her own opinion. Anything goes

i.e., the average score among all eight experts is equal to or higher than 3 (somewhat), then the item will be rated as in the corresponding position. In order to identify students’ epistemological developmental stage, it is preferable that each item only reflect one particular position or possibly its neighboring positions. Therefore, the items that reflect two unrelated positions will be deemed as not useful. For example, items that had high scores on both Multiplicity Subordinate and Relativism are deemed not fitting.

 
Source
< Prev   CONTENTS   Source   Next >