A qualitative case study approach was employed, in which there were seven cases in total from three different layers of the Malaysian education system. In discussing the benefits of qualitative studies, Creswell (2009) argued that they help individuals better understand and explain the meanings that participants attach to the central phenomenon of the research study. In relation to washback, Xie (2015) indicated that a qualitative method is effective in identifying various factors affecting the washback mechanism. As Hughes (1994) and Bailey (1996) have spelled out in their washback mechanism, a test may first influence its participants’ perceptions of and attitudes towards teaching and learning. The present study aimed to look into what specific ways the educational authorities intend to implement PT3 English and how much of their expectations are shared by the stakeholders in the sampled school. In line with such an orientation, a qualitative approach (i.e., interviews) and document analysis were employed to capture the essence of the participants from each level.
The present study intends to capture the insights of key stakeholders within the Malaysian education system. To this end, a purposive sampling method was employed, in which one representative, who is involved in the PT3 English language assessment development at the Malaysian Examinations Syndicate (Dr. R), and another representative from the District Education Office who is involved in monitoring the implementation of the assessment reform (Mr. S), were recruited to answer the first research question of the study. They are referred to as the PT3 English test developers. On the other hand, a principal (Mr. J), an English language panel head (Ms. K), and four PT3 English teachers from one school in one of the states in Malaysia were identified as informants to answer the second research question of this study. Ms. N and Ms. К are junior teachers who have served for less than ten years, whereas Ms. A and Ms. S are senior teachers who have served for more than 20 years. They are referred to as stakeholders in schools. The education ministry ranks the schools’ performances on a continuum of band 1 to band 6. Band 1 refers to schools that have performed excellently on the public examinations, whereas band 6 is considered the lowest ranking. Considering the uncertainties surrounding the present context, at the beginning of implementing the assessment reform, the researchers sampled a band 1 school for their study. Therefore, the middle-banding and low-banding schools can adapt or adopt the findings from the designated school.
Three main documents, namely the secondary level curriculum specifications, PT3 English syllabus, and the teacher guidebook, were scrutinized to learn more about the intended washback of PT3 English. After analyzing the documents, a semi-structured, one-on-one interview was administered with all the respondents in this study. The interview protocol (see Appendix) for the PT3 English test developers and stakeholders in school adopted a parallel design. The questions for the developers were devised upon reviewing the official documents of the Ministry and related literature. In contrast, the questions for stakeholders in schools were devised upon reviewing the official documents and interviews with the developers. All respondents were asked to share their insights on the rationale behind the introduction of PT3 English and the theoretical underpinnings of PT3 English.
Having acquired the approval from the Ministry of Education to carry out this study, the researchers interviewed the PT3 English test developers for their insights. Once their insights were scrutinized, the researchers then interviewed the stakeholders in the selected school. As a confidentiality measure, all the participants involved in this study were assigned pseudonyms. Considering that various participants at different levels were interviewed in the present study, the researchers allowed a leniency to their participants to either use the national language (Malay) or use the English language. The interviews were recorded with the participants’ consent and were transcribed verbatim for analysis.
The insights from the official Ministry documents were scrutinized thematically by zooming in on the theoretical underpinnings of PT3 English and other recommendations. The transcribed interviews were coded and analyzed in NVivo 12. In order to ensure the credibility and trustworthiness of the researchers’ interpretations from the data, data triangulation was employed, whereby data from the interviews with the PT3 English test developers were triangulated with the stakeholders in the sampled school. In addition, the data collection and the interpretation of it was conducted concurrently. Member-checking was also carried out as the researchers handed a copy of the interview transcripts to the participants and had them comment on the issues raised by them during the interviews. Some changes were made to the transcripts upon clarification with the participants.