Simultaneous-Language Item Modelling

To generate construct-equivalent multilingual items and to eliminate the need for designating a source-language group, a third AIG method is available. This method can be described as simultaneous-language item modelling

(Gierl, Lai, Fung, & Zheng, 2016). With this method, the content in the item models is created simultaneously across language groups. Multilingual AIG employs the same cognitive model, meaning the knowledge, skills, and competencies measured by the generated items are designed to be the same across languages. The content in the cognitive model is captured in the item model. When the expression of the target language is followed by the expression of the source language, successive-language item modelling is conducted. When the expression of the source and target languages are produced together, simultaneous-language item modelling occurs. The strengths of the third AIG method include a significant improvement in the rate of item production over that traditional multilingual item development approach, the ability to generate items that are equivalent across languages, and the potential benefit of improved item quality through enhanced conceptual and functional equivalence.

To demonstrate the limitation of successive-language item modelling, Gierl et al. (2015) described the results from a multilingual AIG study in which an English parent item designed to measure operation sense in junior high school mathematics was used to generate equivalent Spanish and Chinese items. The context for the parent item was a pet store which used the verb "wash" (i.e., "Sarah trains and washes cats. She must earn at least $53 each day for w cats she trains and g cats she washes. She receives $5 for each cat she trains and $4 for each cat she washes."). Gierl et al. (2015) created an n-layer model using the contexts of yard work and housework to supplement the pet store context in the parent item. In the process of creating the item model, the phrase "wash windows" and "wash dog" produced different expressions and subsequent translations when Spanish and Chinese were compared because a window is an inanimate object, whereas a dog is an animate object. Because of these differences, the translated content in the item model also differed, thereby producing non-equivalent generated items in Spanish and Chinese. This example demonstrates that when a source language is used to initiate the item modelling step, the target languages can produce non-equivalent generated items when successive translation is used.

Simultaneous-language item modelling helps overcome this problem because only words, key phrases, single sentences, and/or multiple sentences that can be expressed in the same way across the languages are modelled. As a result, idiosyncrasies specific to a particular language

(e.g., idioms unique to a language) or culture (e.g., social norms) can be detected and removed during the early stages of item model development, thereby maximizing linguistic and cultural decentering and leading to improved construct clarity, item relevancy, and item representativeness. To address the problem in the Gierl et al. (2015) example, a verb other than wash would be selected to ensure that a comparable expression between English, Spanish, and Chinese is produced. While successive modelling requires the linguistic expression of a source to target, simultaneous modelling requires a negotiated linguistic expression between source and target where the final item model is only created after the content in each language is agreed upon by the SME and/or translator. Consequently, the item models are designed to be equivalent while the risk of construct bias is reduced, and the degree of linguistic and cultural decentering is enhanced because the source and target language item models are equally open to modification during the development stage. For this reason, simultaneous item development is often preferred (International Test Commission, 2017). The weakness of the simultaneous- language item modelling is its complexity. SMEs and/or translators from each language group are required to work together to create and verify the item model that will be used to generate multilingual items. Also, the languages of interest must be identified early in the test development process so that SMEs who represent each language group can be recruited to participate in all item development steps. This type of vision and anticipation requires a great deal of foresight and planning.

< Prev   CONTENTS   Source   Next >