Previous Frameworks for Examining L2 Learners’ Texts

One limitation of previous studies, as Plakans (2014) observed, is that they tend to focus on the linguistic features of L2 learners’ written texts as indicators of L2 proficiency or development, rather than as indicators of L2 writing proficiency or development (cf. Norris & Manchón, 2012; Polio & Park, 2016). For example, Wolfe-Quintero et al. (1998) reviewed more than 30 studies that analyzed L2 learners’ written texts using numerous measures of syntactic and lexical complexity, accuracy, and fluency (often termed CAF). However, Wolfe-Quintero, Inagaki, and Kim emphasized early on in their book that they “are not interested in measuring the ability to ‘write well’ in a second language, but in measuring language development as it is manifest in a written modality” (p. 2, emphasis added). In other words, their focus was not how L2 writing ability develops, but on the development ofL2 as reflected in specific linguistic features of writing, in this case measures of CAF.

This focus on linguistic features, specifically CAF, can be due to (a) a historical view of writing as a means for teaching and assessing linguistic knowledge (i.e., vocabulary and grammar), rather than a goal in itself, and/or (b) the disagreement that Ellis and Barkhuizen (2005) noted in the field of SLA concerning whether L2 ability includes linguistic knowledge only, or sociolinguistic, discourse, and strategic knowledge as well. Regardless of the reason, most studies on L2 learners’ written texts tend to focus on the linguistic features of CAF (Connor & Mbaye, 2002; Leki et al., 2008; Norris & Manchón, 2012;Plakans, 2014; Polio, 2001; Polio & Friedman, 2017; Polio & Park, 2016). Polio (2001), for example, in a review of studies on L2 learners’ written texts, found that most of the studies focused on linguistic accuracy, linguistic complexity, and lexical features; few studies examined other aspects of writing such as overall quality, content, mechanics, coherence, and discourse features. Similarly, Polio and Friedman (2017) found that CAF was the most commonly used text-based analysis in studies published between 2001 and 2016. Other aspects of writing were examined less frequently. These include formulaic sequences (chunks), cohesion, and paraphrasing and text copying. Plakans (2014) also found that most studies on L2 writing tests tend to focus on linguistic features, specifically CAF, and to a lesser extent on discourse (organization, coherence, cohesion, and content/development). Given the narrow focus of previous studies, Plakans (2014) concluded by calling for future studies to “consider aspects of text that capture the communicative nature of language ... [such as] the interactivity of writing” (p. 10).

The focus on linguistic features ignores other important aspects of L2 proficiency in general, and L2 writing performance in particular, as defined in the literature. As Connor and Mbaye (2002) put it, a focus on linguistic knowledge “fall[s] short of covering the multiple components of writing ability” (p. 265). The Bachman and Palmer (1996, 2010) model of language knowledge, for example, includes four types of knowledge under two main components: (a) organizational knowledge, or knowledge of "how utterances or sentences and texts are organized,” which includes grammatical and textual knowledge, and (b) pragmatic knowledge, or knowledge of “how utterances or sentences and texts are related to the communicative goals of the language user and to the features of the language use setting” (p. 45). Pragmatic knowledge includes functional knowledge (i.e., knowledge of how utterances or sentences and texts are related to the communicative goals of language users) and socio-linguistic knowledge. A focus on linguistic features (i.e., CAF) addresses only one component of the model, that is, grammatical knowledge. The discourse aspects (e.g., cohesion, organization) examined in some studies address textual knowledge. In other words, previous studies tend to focus on organizational knowledge only. Pragmatic knowledge is not usually considered when analyzing L2 learners’ written texts (Connor & Mbaye, 2002; Plakans, 2014; Polio & Friedman, 2017).

This observation was highlighted by Connor and Mbaye (2002) who pointed out that previous L2 studies and commonly used L2 tests of writing tend to focus on the grammatical and discourse (textual) aspects of L2 learners’ texts and to ignore the sociolinguistic and strategic dimensions (i.e., pragmatic knowledge). Connor and Mbaye argued that all four competencies should be reflected in any linguistic analysis of L2 learners’ texts and in the scoring criteria for L2 writing tests.To address this issue, Connor and Mbaye (2002) proposed a Model ofWriting Competence, based on Canale and Swain’s (1980) model of Communicative Language Competence, that covers the four components in the Bachman and Palmer (2010) model: grammatical (e.g., grammar, vocabulary), discourse (e.g., cohesion, text structure), sociolinguistic (e.g., register), and strategic (i.e., audience awareness). The model is shown in Table 2.1.

The Connor-Mbaye (2002) model provides a useful framework for analyzing L2 learners’ texts in terms of the multiple components of writing ability (cf. Barkaoui & Knouzi, 2012). However, the model does not address content or source use. Because a growing number of writing tasks require the use of source texts, a fifth dimension, source use and content, needs to be added to the Connor-Mbaye model.

TABLE 2.1 Connor and Mbaye (2002) Writing Competence Model

Source: Connor and Mbaye (2002, p. 273).

Connor and Mbaye (2002) defined each of the four competences in their model in terms of specific criteria for writing assessment as shown in Table 2.1. For the purposes of analyzing L2 learners’ texts, Barkaoui and Knouzi (2012) redefined each competence in terms of specific criteria, with each criterion operationalized in terms of one or more specific indices. Similarly, McNamara et al. (2014) distinguished between two levels: measures and indices.They used measures to describe “a theoretical construct” such as coherence, and index to refer to the particular way a measure is assessed such as incidence of connectives per 1,000 words. In the following section we propose a framework for analyzing L2 learners’ texts that builds on the Connor-Mbaye (2002) model as well as findings from previous research. This framework distinguishes between four levels of conceptualizing and analyzing L2 learners’ texts as follows:

• 1. Competence: This refers to the four competences as defined in Connor and Mbaye s (2002) Writing Competence Model. This is the highest level of abstraction in our framework.
• 2. Construct: This refers to the sub-components of each competence as defined by Connor and Mbaye (2002), such as syntactic complexity, fluency, and accuracy under grammatical competence. In our framework, a construct is an intermediary level of abstraction, to be operationalized for measurement.
• 3. Measure: Each construct is defined in terms of one or more specific writing measures, such as global complexity and phrasal complexity under syntactic complexity. As such, a measure is a concrete representation of the construct.
• 4. Index: Each writing measure is assessed using one or more specific indices such as mean length of sentence (MLS) for global complexity. An index is a number and, as such, is the most concrete level of representation in our framework.