Role of Task and Proficiency in Changes in Learners’ Writing

As noted above, several of the writing features examined in the study were involved in significant interaction effects (see Table 9.2). These significant interaction effects suggest that the effects of any of the three main variables examined in the study (occasion, task, and ELP group) was moderated by the effects of the other variables. These interaction effects are summarized in this section to address Research Question 4: whether changes in writing features across test occasions are moderated by task and ELP level (see Table 9.2). In addition, there were significant main and interaction effects for students (nested within ELP group) on many of the indices included in the study. These effects point to significant variability across individual students in terms of various writing features. These effects are also discussed in this section.

Occasion by Task Interactions: There were significant medium to large occasion by task interaction effects for ten indices: fluency, complexity by subordination (C/T), clausal complexity (MLC), lexical density, lexical variation (MTLD), lexical sophistication (AWL and academic word list), local coreference cohesion (argument overlap for adjacent sentences), text organization (inclusion of a thesis statement), and metadiscourse (voice ratings). Except for lexical density and lexical variation (MTLD), all the other indices changed significantly after instruction for the independent task, but not for the integrated task. For lexical density and lexical variation, the integrated essays had significantly lower indices after instruction than they did before instruction, while the independent essays had significantly higher indices after instruction than they did before instruction. In other words, after instruction, the participants wrote independent essays that differed significantly from those they wrote before instruction in terms of fluency (i.e., longer essays), various syntactic and lexical complexity measures (i.e., lower complexity by subordination, higher clausal complexity, higher lexical density, and higher lexical variation and sophistication), local and global cohesion (i.e., lower local and global coreference cohesion), text organization (i.e., more thesis statements), and metadiscourse use (i.e., higher voice ratings).The integrated task, in contrast, was not associated with significant changes in eight of these writing features after instruction and was associated with a decrease in lexical density and lexical variation (MTLD) after instruction.

Occasion by ELP Interactions: There were significant large occasion by ELP interaction effects for seven indices: text cohesion (givenness), text organization ratings, register (contractions incidence), metadiscourse (textual ratings, stance, and engagement markers incidence), and source use (percentage of level-2 content units from lecture). Most of these interactions were caused by significant variation across test occasions on these indices for the low ELP groups, but not the medium and high ELP groups. For example, the low ELP groups had significantly lower givenness indices (LSA given/new sentences) than did the other ELP groups before instruction, but after instruction the low ELP groups differed significantly only from the high group, but not the medium group. Text organization ratings seem to have improved for all ELP groups except for the high group. For the incidence of contractions, there was a significant difference across test occasions for the low ELP groups only. The low ELP groups used significantly fewer contractions after instruction than they did before instruction. In addition, the low ELP group used significantly fewer engagement markers after instruction than they did before instruction; the differences across test occasions for the other three ELP groups were not significant. Finally, the low ELP groups included significantly more level-2 content units from the lecture in their essays after instruction than they did before instruction; the changes across test occasions for the medium and high ELP groups were not significant. These findings suggest that the low ELP groups tended to exhibit more changes across test occasions in terms of the six indices listed above than did the high ELP groups.

Task by ELP Interactions: There were significant medium to large task by ELP interaction effects for six indices: clausal complexity (MLC), phrase complexity (NP density), lexical variation (MTLD), lexical sophistication (AWL), lexical bundles (AFL), and metadiscourse (engagement markers). For example, there were significant differences in terms of clausal and phrase complexity across tasks only for the low ELP groups, but not for the medium and high ELP groups. Specifically, the low ELP groups tended to have significantly lower clausal and phrase complexity indices for the independent task than they did for the integrated task. The low ELP group had significantly higher phrase complexity and AWL indices for the integrated task than did the other ELP groups. Additionally, the low ELP group had significantly lower lexical sophistication (AWL) indices than did the other ELP groups for the independent task, but significantly higher AWL indices than each of the other ELP groups for the integrated task. These patterns suggest that the low ELP group copied more noun phrases and long words from the reading text in the integrated task than did the high ELP groups. For lexical variation (MTLD), differences across tasks were larger for the high ELP group than they were for the other ELP groups. Specifically, the high ELP group had significantly higher MTLD indices for the independent task than they did for the integrated task. These patterns suggest that participants with lower ELP tended to exhibit more variability across tasks in terms of the indices above than did participants with higher ELP, except for lexical variation.

Individual Variability: Finally, there were significant interaction effects of occasion by task by ELP level on global complexity (MET) and complexity by subordination (C/T) as well as significant main and/or interaction effects for student (nested within ELP group) on 28 indices in the study. For example, there were significant student (within ELP) main effects on measures of fluency, linguistic accuracy, syntactic complexity, lexical complexity, local cohesion, and source use. In addition, there were significant occasion by student and task by student interaction effects on MLT, AWL, coherence ratings, local cohesion, and text organization ratings.These significant main and interaction effects for student suggest that students within ELP groups varied significantly in terms of measures of various writing features and that they exhibited significantly different patterns in terms of changes across occasion and differences across tasks in terms of these writing features. This was true for essay scores as well. That is, students within ELP groups varied significantly in terms of their writing scores and individual students exhibited significantly different patterns in terms of score changes across test occasions and score differences across tasks.

The significant interaction effects between occasion, task, ELP group, and student (within ELP group) indicate that some students made significant gains on some of the indices across test occasions, while others did not. The main effects of occasion, task, and ELP group on the writing indices examined in the study summarized above represent effects at the group level. The significant interaction effects indicate that changes in some of the writing indices across test occasions depended on task type, proficiency group, and/or individual students. Some changes occurred for all ELP groups and both tasks (e.g., register ratings), while the patterns of change in other writing features depended on task type (e.g., lexical variation) and/or ELP group (e.g., ratio of engagement markers); for other writing indices (e.g., local cohesion), as well as essay scores, the patterns of change varied across individual students within ELP groups too.