“I Was Not Thoughtful Enough Before”: Exploiting the Learning-Oriented-Assessment Potential of a Test-Taker-Oriented Rubric for Summative Assessment
Beverly Baker, Sebastien Polikar and Maryam Homayounzadeh
Assessment plays a key role in both fostering learning and the certification of students. However, unless it satisfies the educational purpose of ensuring students can identify high-quality work and can relate this knowledge to their own work, the likelihood that they will attain high- quality themselves is much reduced (Boud & Associates, 2010, p. 1).
Our primary formative activity in our classes, as in academic English writing classes all over the world, has consisted of qualitative comments on our students’ drafts as they navigate the writing process. This activity, while very useful, does not provide for increased learner agency, keeping them in a passive, reactionary mode of thinking (Mak, 2019). As ESL writing instructors at the post-secondary level, we have long observed this lack of agency in our students: while they were often motivated to improve their writing and saw the importance of an academic writing skill for academic success, they were not adept at setting goals for their own learning or even explaining the learning outcomes of the course we were in together. In addition, they appeared to be largely unaware of the criteria by which their work is judged.
The following is a report on our attempt to maximize our students’ language learning, in addition to their autonomy, their ownership of the learning process, and their capacity for critical reflection on their work, by moving beyond formative feedback on drafts. Specifically, we report on an assessment activity we developed called “You Be the Grader,” which makes use of a test-taker version of an official rubric for a high-stakes academic writing assessment that most of our students have taken or may take in the future. In the assignment, students intensively study the rubric, then they practice grading blinded peer writing samples and collaborating with their classmates to defend and negotiate the final grades.
In this chapter, we will share details of this assignment and how it was carried out in the classes, as well as our discoveries from our students’ work. We have found that it has been very useful and productive to frame these discoveries in terms of learning-oriented assessment (LOA), so we will first situate our project within the discourse of LOA. More specifically, we will describe the key features of LOA as well as how an LOA viewpoint allows for the blurring of boundaries between formative and summative assessment practices in the classroom. We will also review the practice of peer assessment as viewed within LOA.
As the name suggests, LOA refers first and foremost to an orientation to assessment rather than to a specific tool or activity - all tools and activities are conceived and used for supporting learners and learning, rather than simply for “recording student achievement” (Turner & Purpura, 2016, p. 255). In this view, assessment is at the service of learning and is explored for its use in supporting learning processes. Boud and Soler (2016) reiterate this in their discussion of “sustainable assessment,” when they state that in LOA, assessments are conceptualized in terms of their learning outcomes - as well as their capacity for facilitating learners’ continuous development beyond the classroom.
LOA draws on research in formative assessment (Black & Wiliam, 1998), which has considerable conceptual overlap with the term assessment for learning (Assessment Reform Group, 2002). The concept of assessment as learning is also pertinent, referring to the use of assessment by learners to better understand expectations and to subsequently enhance metacognitive activities such as independent goal setting (see Zeng et al., 2018). Dann (2002) defines assessment as learning as “a process through which pupil involvement in assessment can feature as part of learning” (p. 153). LOA is not viewed as being in competition with these other concepts, but as a continuation of the conversation, as researchers attempt to conceptualize the role of assessment activities within theories of learning. Carless et al. (2006) used the term “learning-oriented assessment” to refer to an approach to classroom assessment in which advancement of student learning comes first and is thus emphasized over measurement purposes (Carless, 2007,2015; Carless et al., 2006). Important features of LOA include involving students in conversations about constructs of and expectations for assessment, and providing activities to promote learner self-regulation - empowering students to continue their learning independently outside the classroom, and providing them with metacognitive tools to plan for, monitor, and evaluate their own learning.
This focus on metacognitive regulatory processes is seen as essential in linguistic development: Purpura (2014) states that “metacognitive processes regulate the cognitive ones related to stages of processing and of generating output in task performance” (p. 15). In other words, and specifically related to academic writing, metacognitive strategies (such as identifying the key points in source documents and planning their effective use in an essay) can work to regulate cognitive processes (such as selective attention when reading relevant sources, or recall of reporting language when referring to the source). In addition, improved selfregulation can cultivate lifelong learning (Clark, 2012; Shepard, 2000). According to self-determination theory, (Ryan & Deci, 2017a, 2017b, 2020), motivation which is based on student autonomy is of better quality than motivation from external sources, in that it is more self-sustaining and predicts better educational outcomes. As students themselves identify tools to improve their own learning, agency is enhanced and so is motivation, as they observe that their progress is under their own control.
Three guiding principles have been established for LOA: (a) assessment tasks as learning tasks; (b) student involvement in assessment, e.g., as peer- and/or self-evaluators; and (c) timely and forward-looking feedback to enhance their evaluative expertise (Carless et al., 2006; Carless, 2007). Our project explores the first two of these principles in its design of a peer-evaluation assessment task, primarily for a learning purpose. Turner and Purpura (2016) have developed a working framework for LOA, conceived of as a set of interrelated dimensions:
- • Contextual dimension;
- • Elicitation dimension;
- • Proficiency dimension;
- • Affective dimension;
- • Interactional dimension;
- • Learning dimension;
- • Instructional dimension.
As we provide more details of our “You Be the Grader” assignment below, we will demonstrate its place within the proficiency dimension, the interactional dimension, and the learning dimension. Regarding the proficiency dimension, the “You Be the Grader” assignment is designed to allow students insight into the understandings and values of teachers regarding proficiency in academic writing - i.e., the implicit model of L2 writing as operationalized in the rating scale and enacted in the classroom. This transparency is inherent in LOA approaches, where “learners should have a clear understanding of what it is they are expected to learn and what the criteria for success are” (Turner & Purpura, 2016, p. 264). Regarding the interactional dimension, we have designed the activity so that students take on the role of grader and demonstrate not only their knowledge of quality academic writing but also gaps that indicate to instructors the direction of future instruction. Regarding the learning dimension, we are aiming with this assignment to promote self-regulation, where students are more in control of their own learning by being given the opportunity to reflect on peer work and on their own process of critically engaging with the rubric.