Approaches to Researching Learning-Oriented Assessment in Second and Foreign Language Classrooms
Heidi Liu Banerjee
Learning-oriented assessment (LOA) has gained considerable attention in second and foreign language (L2) research over the past two decades due to its close connection to learning and teaching. Greatly influenced by the theoretical and pedagogical foundations laid by classroom-based assessment (CBA) research and practices, LOA in classroom settings has been found to encompass several types of CBA with a similar learning-centered emphasis, such as “formative assessment,” “assessment for learning,” and “dynamic assessment” (Leung, 2020). While an abundance of research has examined how assessment can facilitate learning and teaching in L2 classrooms, Turner and Purpura (2016) pointed out that there has been little systematicity with regards to the approaches to pursuing classroom- based LOA research, most likely due to the complexity of the dynamics between learning, teaching, and assessment in L2 contexts.
Seeing the need to address the complex dynamics of classroom-based LOA from both learner and teacher perspectives and to further highlight learning goals, learning processes, and learning outcomes embedded in L2 CBA, Turner and Purpura (2016, see also Purpura & Turner, 2016, Purpura & Turner, 2021; Saville, Chapter 2 in this volume) proposed an LOA framework that consists of seven interrelated, yet independent, dimensions of teaching, learning, and assessment: the contextual dimension, which concerns the real-life language use domains in which learners would realistically demonstrate their L2 knowledge, skills, and abilities (KSAs); the proficiency dimension, which relates to the targeted L2 KSAs learners are expected to demonstrate based on the curriculum, instruction, or standards; the elicitation dimension, which is concerned with how evidence of learners’ L2 KSAs can be elicited to reflect their expected proficiency; the socio-cognitive dimension, which relates to the cognitive, socio-cognitive, and strategic demands of assessment tasks, the promotion of learning through feedback and assistance, learners’ ability to automatize the learning targets as facilitated by the assessment processes, and the conceptualization and operationalization of learning; the
instructional dimension, which concerns how teachers, peers, or learners themselves utilize the information from planned or unplanned assessment to provide assistance or feedback, and to promote further processing and learning; the socio-interactional dimension, which relates to the interactions between teachers, peers, and learners that are embedded in instruction within an assessment; and finally, the affective dimension, which is concerned with how assessments tap into learners’ socio-psychological dispositions (Purpura & Turner, 2021). Together, these seven dimensions illustrate the complexity and multidimensionality of L2 classroom-based LOA, and this LOA framework appears to provide a promising guideline for robust investigations into LOA in L2 classrooms.
The complexity and multidimensionality of L2 classroom-based LOA have posed numerous challenges for researchers. In particular, how learning actually transpires through assessment, planned or unplanned, and how various LOA dimensions contribute to learning have not always been fully captured in L2 classroom contexts. As Jones and Saville (2016) stated, the key to conducting robust LOA is being able to demonstrate evidence for and of learning; however, this is not always accounted for in LOA research, perhaps due to the difficulty of capturing “learning” comprehensively. The purpose of this chapter is, thus, to examine the research methods that have been used to conduct LOA research in L2 contexts through the lens of the LOA framework proposed by Turner and Purpura (2016), and to explore the evidence of and for learning that has been presented in current studies. Then, recommendations of the principles of conducting LOA research in L2 classrooms are provided.
Approaches to Researching LOA in L2 Contexts: What Has Been Done?
To investigate the current approaches to conducting LOA research in L2 contexts, a total of 17 studies were surveyed. The following inclusion criteria were used when selecting the studies:
- 1. The study was conducted in L2 contexts but not restricted to L2 classrooms (i.e., including both CBA and large-scale assessments); AND
- 2. The study is empirical in nature (i.e., presents empirical data); AND
- 3. The study directly indicates LOA as its key component by having the term “learning-oriented assessment” in its title or abstract or by discussing LOA extensively in its main text; OR
- 4. The study was presented at the 2014 Teachers College Columbia University Roundtable on LOA in Classroom and Large-Scale Assessment Contexts. If the study has been published, the published version was examined; if not, the presentation version was examined.
It should be noted that the studies included in this chapter are in no way exhaustively representative of all LOA studies in L2 contexts. For example, dissertations were not considered during the search process. In addition, if multiple studies were found to use the same dataset which involved the same data collection and analysis procedures, only the most recent study was included (e.g., May et al., 2020; Nakatsuhara et ah, 2018). The 17 studies included for further examination are marked with an asterisk in the reference.
The next step was to identify the research approaches used in these studies. Information regarding how data were collected and analyzed was first extracted. Then, following Jones and Saville (2016), how each study presented its evidence for and of learning was identified. Specifically, evidence for learning refers to any instruction, task, assessment, or interaction that created learning opportunities, and evidence of learning refers to the way in which actual learning was captured in the study. Finally, based on the purpose of the study or the research questions, each study was coded for the LOA dimensions it focused on (Purpura & Turner, 2016, 2021; Purpura, 2016). The coding of the LOA dimensions was done twice within a one-week interval to maximize intra-coder reliability. Table 4.1 presents the summarized information.