Content Coding for Item Generation

With the availability of content codes, appending this metadata to each individual item is a tedious, time-consuming, and laborious task. SMEs must first locate the appropriate content for an item in an ad hoc content list, specification, or framework and then map these codes onto each individual item. This process may also require an additional review step by an independent group of SMEs to ensure that the appropriate content code is selected and that the content codes are consistently applied to items. Content coding in AIG is substantially simplified because the unit of analysis shifts from the item to the model, meaning that the SME appends the codes to the elements and/or values in the item model. These codes are then assembled across all of the elements and/or values during item generation in step 3 to produce a unique content code list for each generated item. Model rather than item coding simplifies the process because the SME only needs to assign content to the model. These model-based codes, in turn, are assembled into a list of content codes during the item generation step so that specific descriptive information is created for each item. The result of this process is that content codes can be easily added to generated items, thereby producing metadata that can be used to organize and manage the content in the bank.

It is important to emphasize that the item model contains all of the content that is necessary for generating items. Hence, content coding the item model can be characterized as adding a new dimension of information to this model, as will be demonstrated later in the chapter. Three different types of metadata can be added to this new dimension. The first type of metadata requires coding at the model level. This type of metadata is the most general. With model-level coding, specific codes that describe all of the generated items from a particular item model are used. For example, an item model that is designed to generate math items could contain a mathematics content code. As a result, all items generated from this model will be coded as mathematics. The second type of metadata requires coding at the element and/or value level. This type of metadata is the most specific. Recall that a cognitive model contains features in the bottom panel. Each feature specifies two nested components. The first nested component for a feature is the element, and the second nested component for a feature is the constraint. Elements contain content specific to each feature that can be manipulated for item generation. The content in each element is stored as values. These values can be denoted either as string or integer values. Because each generated item requires a unique combination of values for each element, coding the values will also produce a unique combination of content codes. For example, if a value is associated with the presentation of a fever, then that value can be used with the content code of fever. Following this logic, SMEs can add content codes that are unique for each value, resulting in generated items that contain unique codes that correspond to the values displayed in the item. Defining content codes at the element and value levels has enormous benefits for organizing and managing a bank because each generated item could have a unique content code, meaning that all of the generated items in the bank could be differentiated from one another. The third type of metadata—which is unique for the selected-response item type—requires coding the options. This type of metadata is often the most descriptive. Model- and element/value-level content coding can be used to describe any type of generated item with a stem. But when the item type also includes options, metadata can be applied to this information source as well. The correct and incorrect options can be assigned with model and element/value-level content codes.

These options can also be assigned rationale codes (see Chapter 10). Thus far, we have described a process in which the item model is content coded. The item model, in turn, compiles these codes into a list that is applied to each generated item. But rationales, which serve as brief descriptions of how the item should be solved, can also be included as metadata and assigned to the correct option. For example, if the correct option requires knowledge of how to multiply with odd numbers, then a rationale code with this description can be appended to the correct option. Similarly, if one of the incorrect options is based on a misconception of ratios (see Chapter 5, Table 5.1), then a distrac- tor code with this description can be appended to a specific incorrect option. Taken together, content coding at the model, element/value, and option levels provides the SME with a comprehensive and flexible approach for adding metadata to each generated item.

< Prev   CONTENTS   Source   Next >