Advanced Methods in Automatic Item Generation: Theoretical Foundations and Practical Applications


The Problem of Scaling Item DevelopmentAutomatic Item Generation: An Augmented Intelligence Approach to Item DevelopmentBenefits of Using AIG for Item DevelopmentPurpose of This BookReferences: Cognitive Model Development: Cognitive Models and Item GenerationBenefits of Using Cognitive Models For AIGDeveloping Cognitive Models for AIGA Word of Caution When Creating Cognitive ModelsTwo Types of Cognitive Models for AIGLogical Structures Cognitive ModelKey Features Cognitive ModelReferences: Item Model Development: Template-Based AIG Using Item ModellingLayers in Item ModelsItem Generation With 1-Layer Modelsn-Layer Item ModelsItem Generation With n-Layer ModelsTwo Important Insights Involving Cognitive and Item Modelling in AIGNon-template AIG: A Review of the State of the ArtIs It Preferable to Use a Template for AIG?NoteReferences: Item Generation: Approaches for Generating Test ItemsThe Importance of Constraint CodingLogical Constraint Coding Using BitmaskingDemonstration of Item Assembly Using the Logical Constraints ApproachLogical Structures Cognitive ModelKey Features Cognitive ModelItem Assembly Using the Logical Constraints ApproachReferences: Distractor Generation: The Importance of the Selected-Response Item in Educational TestingThe Contribution of Distractors in the Selected-Response Item FormatTraditional Approach for Writing DistractorsMethods for Distractor GenerationDistractor Generation With RationalesDistractor Pool Method With Random SelectionSystematic Distractor GenerationNoteReferences: Putting It All Together to Generate Test Items: OverviewMathematics Example Using the Logical Structures ModelCognitive Model DevelopmentItem Model DevelopmentItem Generation Using Constraint CodingSystematic Distractor GenerationA Sample of Generated Math ItemsMedical Example Using Key FeaturesCognitive Model DevelopmentItem Model DevelopmentItem Generation Using Constraint CodingSystematic Distractor GenerationA Sample of Generated Medical Items: Methods for Validating Generated Items: A Focus on Model-Level OutcomesSubstantive Methods for Evaluating AIG ModelsCognitive and Item Model Review Using a Validation TableDistractor Model Review Using a Validation TableSubstantive Model Review Using a Rating ScaleSubstantive Methods for Evaluating AIG ItemsAIG versus Traditional Item Review: Item QualityAIG versus Traditional Item Review: Predictive AccuracyStatistical Methods for Evaluating AIG ItemsStatistical Analyses of the Correct OptionStatistical Analyses of the Incorrect OptionsCosine Similarity Index (CSI)The Key to Validating Generated ItemsReferences: Content Coding: Challenges Inherent to Managing Generated Items in a BankManaging Generated Items With MetadataContent Coding for Item GenerationAssembling Content Codes in Item GenerationContent Coding ExamplesLogical Structures Mathematics ModelKey Features Medical ModelReferences: Generating Alternative Item Types Using Auxiliary Information: Expanding the Expression of Generated ItemsGenerating Items With SymbolsGenerating Items With ImagesGenerating Items With ShapesChallenges With Generating Items Using Auxiliary InformationReferences: Rationale Generation: Creating Rationales as Part of the Generation ProcessMethods for Generating RationalesCorrect OptionCorrect Option With RationaleCorrect Option With Distractor RationaleA Cautionary Note on Generating Solutions and RationalesBenefits and Drawbacks of Rationale GenerationReferences: Multilingual Item Generation: Beyond Monolingual Item DevelopmentChallenges With Writing Items in Different LanguagesMethods for Generating Multilingual Test ItemsLanguage-Dependent Item ModellingSuccessive-Language Item ModellingSimultaneous-Language Item ModellingExample of Multilingual Item GenerationValidation of Generated Multilingual Test ItemsReferences: Conclusions and Future DirectionsIs AIG an Art or Science?Is It “Automatic” or “Automated” Item Generation?How Do We Define the Word “Item” in AIG?How Do You Generate Items?What Is an Item Model?How Do You Ensure That the Generated Items Are Diverse?How Should Generated Items Be Scored?How Do You Organize Large Numbers of Generated Items?What Does the Future Hold for Item Development?ReferencesAuthor IndexSubject Index