The Instructional Context

The context for this project is an ESL credit-bearing course in a bilingual French-English university in Canada. Students in French-language programs may choose to take this course as an optional elective course, or they may take it as part of a major or minor in English as a Second Language. However, the vast majority of students enrolled in this course are international students from all disciplines, mandated to take this course as a condition of their admission to the university. The objective of this course, as taken from the official course description, is “development of effective, appropriate and accurate academic writing skills.”

The “You Be the Grader” Assignment

We developed this assignment to allow students to engage with both individual cognitive processes (as they read the peer essays and compared them to level descriptions in the rating scale) and socio-cognitive processes (as they discussed with colleagues to justify and negotiate grades in terms of the scale). The learning objectives and the evaluation criteria for this assignment were presented to the students as the following:

  • By the end of this assignment you will be able to apply grading criteria in a systematic and defensible manner in the grading of writing.
  • You will also be able to critically reflect on the challenges involved in grading writing in a reliable and practical way, that allows for valid interpretations of the students’ abilities.

Evaluation Criteria: You will NOT be evaluated on whether you get the same grades as the official grades! You will be evaluated on completing all elements of the assignment in clear and accurate language, showing clear evidence of analysis of the essays according to the criteria in the rubric; and showing critical reflection on your experience, in a way that demonstrates understanding of key course concepts.

Authors 1 and 2 are instructors of these ESL advanced academic writing classes, and made use of the test-taker-oriented rubric developed by Authors 1 and 3 to develop this in-class assignment. The assignment was split into two parts: in-class grader training and out-of-class homework.

Training (In Class)

Day 1: In class, students were given the assignment instructions, a rubric, and some sample essays. The prompt for the essays was the following: If you could travel in time, where and to what time period would you go? Pick one specific time and place, either in the future or in the past, and explain why you would go there. Support your choice with clear reasons or examples.

This prompt was chosen as it was typical of essay prompts for the assessment, which eschews the traditional five-paragraph essay structure and typically involves relating a personal story or reflection to a larger theme. It also is designed to be relevant to students of all cultural and disciplinary backgrounds. As a whole class, we spent approximately 20 minutes reading over the rubric and accompanying information. Students asked questions about the vocabulary and various criteria contained in the rubric. The entire rubric, as well as the process for its creation, can be found in Baker et al. (2020), but Table 10.1 contains two levels of the rubric for reference purposes, to get an idea of the criteria and language contained in the rubric.

As a class, we spent the next 30 minutes or so reading over one of the sample essays and comparing it to the rubric in order to determine the grade that it should receive. Specifically, we discussed elements of the text as described in the scale, such as identifying the thesis and deciding whether there was ample and relevant support and whether they were convinced by the writer’s argument or not. We also looked at how the essay was organized, with links made to guide the reader through the argument. We also discussed the language in terms of grammar and vocabulary, their effects on comprehension, and the use or overuse of simple and cliched writing. We discussed the range of structures used by the students and the frequency and types of errors made. Finally, we came up with a grade for the essay based on all of the criteria we had discussed.

After this, students spent another 20-30 minutes working individually on another sample. They were asked to decide on a grade for the essay and to note at least three reasons (based on the evaluation criteria) to justify the grade they had given the essay.

Day 2: Students spent the first ten minutes comparing their grades and justification for the essay they graded in the previous class and discussing their reasons for agreeing or disagreeing with their colleagues on the grade it should have received. Then, following a “think-pair-share” format (Lyman, 1981), students individually analyzed and decided on provisional grades for three or four more essays, then shared these grades with partners, then in larger groups (three to four students) to attempt to come

“I Was Not Thoughtful Enough Before” 169

Table 10.1 Two levels of the seven-level CanTEST test-taker-oriented rubric


Modest writer

• Argument and Support: Your main ideas are usually clear and have support, but the reader sometimes needs extra effort to understand your point. The reader might be convinced by your argument.

• Organisation: Your ideas are pretty easy to follow, but the reader might need to re-read sections once or twice. You might have over-used or mis-used some basic linking devices (finally, therefore, etc.) which cause the reader to re-read a sentence or two to make sure they understand.

• Vocabulary: Your word choices and word forms (e.g., nouns or adjectives) are usually accurate but not very original or with much variety; you may have relied too much on the words in the question or on some cliches or ready-made structures.

• Language structures: You use simple grammatical structures effectively; you try some more complex structures; sometimes they are right but sometimes they are not, and can cause small misunderstandings.


Somewhat limited writer

• Argument and Support: Some of your main ideas are clear, but maybe one or two are not clear or relevant to the topic. Some ideas have support, but others have no support or weak support. The reader might not be convinced by the argument.

• Organisation: It is often not easy to follow you in the text because the relationship between ideas is not so clear; maybe you used the wrong linking devices. Readers need to re-read sometimes to double-check meaning.

• Vocabulary: You used a limited range of vocabulary, not enough for the task or for professional academic writing; you have used some words incorrectly or used ready-made structures instead of original writing.

• Language structures: Most of your sentences have simple structures; some errors make it difficult for the reader to understand your meaning.

to a consensus. We then opened up the discussion to the whole class, and each group took turns sharing their grades and talking about what they had discussed within their respective groups. Finally, we went over the official grades for these essays and engaged in further discussion to try to account for any discrepancies.

Part Two: Homework/Evaluation

Students were given five more essays to read at home individually, as well as a grading feedback form to fill out where they were asked to provide written comments on each of the criteria from the rubric (argument and support, organization, vocabulary, and language structures). An example of written comments was provided as a model (for the assignment feedback form see Baker et al., 2020). In addition to filling in this form, students were given a set of questions that prompted them to reflect upon how the assignment would help them in their own writing and revising work. Students had one week to complete this assignment, which was worth 15 percent of their grade for the term.

< Prev   CONTENTS   Source   Next >