Designing Feedback Materials and Activities for IC: An Illustration
As shown in this chapter so far, the theoretical insights and empirical findings of feedback practices have become increasingly robust. There is now research evidence for features of effective and ineffective feedback, and a strong argument for the role of learner engagement and feedback literacy. However, there is also a need for more work that exemplifies how to apply these principles to teaching/learning contexts and materials development. Moreover, as mentioned at the beginning, research and practitioner-oriented work on providing feedback for IC has been generally under-represented. This chapter represents an effort to fill these gaps.
In the following, I will present two sets of LOA feedback materials developed in two Cambridge Assessment English funded research projects, namely, (a) a checklist ofIC features with accompanying description for teachers and feedback to learners (Nakatsuhara et al., 2018), and (b) a resource bank of worked examples for the features on the IC checklist (Lam, 2019). These materials will be used to illustrate how the principles discussed above (Figure 6.1) may be (and have been) applied in designing feedback materials and activities.
The IC checklist was developed with the aim of providing LOA feedback to learners about features of IC in paired discussions. It was designed to be used by teachers and learners preparing for Cambridge Assessment English’s B2 First (formerly known as FCE) speaking test. The checklist identifies particular IC features in learners’ performance in the Collaborative Task (Part 3), but is also adaptable for use with other paired/group speaking tasks. In the current format (since 2015) of the collaborative task, the pair of candidates are given a topic and some ideas to discuss for about two minutes, e.g., how the six ideas given in the task prompt make life in the city more enjoyable. The candidates will then have about one minute to decide which idea(s) is/are the best or the most important. Performance is rated on four criteria:1 grammar and vocabulary’, discourse management, pronunciation, and interactive communication.
The research that underpinned the development of the IC checklist and feedback material involved (a) collecting stimulated recall comments from six experienced B2 First speaking examiners on 12 paired interaction videos; (b) thematic analysis of the examiner comments with NVivo 11; and (c) a focus group with four experienced teachers trialing the use of the checklist (see Nakatsuhara et al., 2018 for a detailed account). Two
Figure 6.2 Extract of IC checklist (full version). Source: Nakatsuhara et al., 2018, p. 62.
versions of the IC checklist were compiled: a full version and a concise version. The full version of the IC checklist comprises nine categories, with several micro-features within each main category (see Figure 6.2 for an extract, and Nakatsuhara er al., 2018, pp. 62-66 for the full checklist).
- 1. Starting the discussion and contributing new ideas;
- 2. Responding to the partner;
- 3. Maintaining and developing the interaction;
- 4. Negotiating towards a common decision;
- 5. Providing or needing support;
- 6. Interactive listening;
- 7. Body language;
- 8. Using functional language for interaction;
- 9. Interacting confidently and naturally.
The concise version of the IC checklist consists of a one-page checklist and a one-page feedback text with four main categories, designed for easier processing by the teacher when conducting classroom assessments of the paired discussion task (see Figure 6.3 for an extract, and Nakatsuhara et al., 2018, pp. 67-68 for the complete, concise checklist).
The IC checklist and the accompanying feedback illustrates three of the principles for feedback practices: providing feedback that is (I) specific, (II) forward-looking, and (III) at an appropriate level. Extract 1 below displays a micro-feature under the main category 2) responding to partner, with 2c) and 2e) being the corresponding positive and negative features, and the relevant feedback for a learner who “needs more work” in this area:
94 Daniel M. К. Lam
Extract 1: Extract of IC checklist (full version)
2c) link rheir own contribution to the partner’s (developing an idea across turns)
2e) only give their own ideas and opinions rather than link to what the partner has said (and develop that idea further)
Needs more work -» To achieve a successful discussion, it’s very important to listen to what your partner says and respond and/or add to his/her idea. Try to balance between offering your own ideas and responding to
Figure 6.3 Extract of IC checklist (concise version). Source: Nakatsuhara et al., 2018, p. 67.
your partner’s ideas. When responding, don’t simply say you agree or disagree and stop there. Give a reason, an example, or other comments on your partner’s idea before introducing a new idea. Some useful phrases to link your own ideas to what your partner has said are:
Like what you said, ...’
‘As you mentioned,...’
‘I agree with your idea that ..., because....’
(Nakatsuhara et al., 2018, p. 62)
Principle I. Specific: As an LOA feedback tool, the IC checklist was designed to provide learners with feedback on specific IC features in their paired interaction. The checklist, developed from the thematic analysis of examiner comments, can identify more IC features in learners’ performance than the rating descriptors. Through using the checklist in observing or evaluating a paired discussion, teachers can raise learners’ awareness of different strengths or weaknesses (e.g., link their own contribution to the partner’s vs. only give their own ideas) in their IC performance. Therefore, the checklist offers more nuanced information about the learners’ current performance than a score of 3/5 or its level descriptor (e.g.,‘initiates and responds appropriately’), giving it potential for use as a diagnostic tool in classroom assessment.
Principle II. Forward-looking: With weaker aspects of the current performance already identified on the checklist itself (e.g., 2e), the accompanying feedback text is crafted with a forward-looking direction (Carless, 2007; Wiliam, 2010), outlining ways to improve on this aspect. As shown in Extract 1, the feedback for “responding to partner” gives learners a suggestion for future performances (e.g., “Try to balance between offering your own ideas and responding to your partner’s ideas”) and the means to achieve this (e.g., “Give a reason, an example, or other comments on your partner’s idea before introducing a new idea”). It also provides some useful phrases learners can draw on (e.g., ‘Like what you said, ...’) when they do the collaborative task again.
Principle III. Appropriate level: The feedback is presented in a manner and at a level of detail appropriate to the learners (Dunlop, 2017; Hattie & Timperley, 2007). While the feedback draws on the examiners’ comments and recommendations and the description of the relevant IC feature (see the middle column of Figure 6.2), the text was written and further revised in ways sensitive to the learners’ proficiency level (at or below CEFR B2 level) - with simplified sentence structure, reduced length, and technical terms removed or paraphrased. This was done to facilitate learners’ understanding of the feedback, which, as discussed above, is a prerequisite for learners to act on the feedback (Sadler, 2010; Steen-Utheim & Hopfenbeck, 2019).
Building on the IC checklist and feedback material (Nakatsuhara et al., 2018), Lam (2019) developed a resource bank of worked examples for the
IC features in the checklist and feedback material, applying the principles of (IV) relating to success criteria, (V) engaging the learner in a proactive role, and (VI) promoting self-regulated learning. The key considerations for enhancing such LOA feedback are how to foster learners’ understanding of the (sometimes abstract) success criteria; how to engage learners in more active and deeper processing of the IC features; and how to equip learners with the skills to evaluate their own performances or their peers’ (cf. making judgments in the feedback literacy framework by Carless & Boud, 2018).
Taking account of the above considerations, the resource bank was designed to (1) optimize understanding of the IC checklist feedback by illustrating the IC features with examples and in context; and to (2) provide a framework for a discovery learning activity (“analyzing exemplars”), in order to raise learners’ awareness of the success criteria and train them in evaluating IC performance in paired discussions. The process of developing the resource bank involved two stages: (a) identifying IC features and feedback text in Nakatsuhara et al. (2018) where illustrative examples would be beneficial, and (b) developing the worked examples through transcribing and analyzing the 12 paired interaction videos following a conversation analytic approach. Overall, a total of 15 worksheets with 33 examples were produced (see Lam, 2019, for more details). Each worksheet consists of three parts:
- • Part 1: Simplified transcript extract(s) illustrating an IC feature, and web links to the paired interaction videos where publicly available (Extract 2);
- • Part 2: Guiding questions (Extract 3);
- • Part 3: A “lesson learned” narrative (Extract 4).