Subject Index

applying content codes 150 assembling content codes 153 AIG validation 120; cognitive and item model 122-126; computational linguistics 141; substantive 202; validation table 122-123, 126, 141 Augmented intelligence 6-7, 9 Automated Essay Evaluation 221 automatic item generation (AIG) vii-x, 6-9, 11-15, 24-26,28-30, 43, 55-57, 79, 82-83, 85, 120-124, 131-134, 147-148, 151, 159-160,

189-191, 206-217, 220; and cost-benefit 1,6, 12, 189, 221; computer system 222; content and workflow 124; development process 142; modelling process 13, 25, 101, 120-122, 160, 183, 208, 211,213; non-template-based AIG 57-58, 67; statistical 134-138; and test security benefit 13, 214; theory 124, 126, 209; and traditional items 131-133

bit-masking 66, 70-74, 78-79, 114, 213 black box 7-8, 210

black box description 121,210 Bloom's taxonomy 27 blueprints 42, 149 Boolean case logic 68-70, 73, 78 b-parameter 135, 202

classical test theory (CTT) 134-135, 138, 202; classical test theory framework 202 classroom assessment 5 cloze test format 67 cognitive and item model review 122 cognitive model 9-12, 14-15, 23-25, 27-46, 51-52, 54-57, 73-74, 77-78, 86-87, 91-93, 96-97, 101-103, 109-112, 121-127, 176, 181, 190, 192, 210-213, 215 cognitive model development 14, 23- 39, 101, 109, 220; cognitive model benefits 25-26; cognitive model content 50, 121; cognitive model evaluation 28; cognitive model quality 29; cognitive representations 26; key features cognitive model 36-37, 44-45, 54, 96-97, 125,

127; logical structures cognitive model 30-35, 45, 68, 74, 125

computer-based testing (CBTj vii-viii, 1-4, 62, 175, 187; architecture and systems vii, 187; assembly, computer-based 9 confirmatory AIC 215 constraint coding 67-68, 75 content code 15, 103, 147-158,

207, 212, 215-216; content review 148; content coding, ad hoc 150; content coding examples 154 content specialists 4, 215 conventions guide content development 62

correct option 10, 29, 42-44, 48-49, 81-89,91,93-94, 96-98, HI- 112, 116, 123-127, 134-138, 140-141, 153, 177-184 Cosine Similarity Index (CSI) 1, 6, 13, 134, 138-141,213

data engineering 11 differential item functioning (DIF) 99, 203-204

distractor development in AIG 83-87, 89, 91,96, 126-127; distracters 42, 81; distractor analysis 135—

  • 137; distractor constraint 48,
  • 165; distractor rationale method 182

educational achievement test 3 educational measurement 175, 209 educational testing practices 219 evaluating AIG Items 131, 134 evaluating AIG Models 122 evaluating Generated Items 130 examples of multilingual AIG 193 exploratory AIG 215 exploratory factor analysis 60, 203, 215

factor analysis 60, 203; confirmatory 203, 215; multiple-group confirmatory 203; factor loadings 203

feedback 2, 28-29, 155, 175-178, 181-184, 220; comprehensive 183; formative feedback 2, 176 field-tested items 216 field test forms 138 Formative assessment 2, 61, 175 formative feedback 2, 176

item formats 14-15, 61-62, 66, 82, 159, 207, 212-213 item model 11-15, 45-61,68-77, 81-98, 101-114, 120-36; n-layer item models 49-54; 1-layer example 51, 101; 1-layer math model 108, 140; 1-layer model 46, 49-51, 53, 118, 133, 138, 140-141, 177, 194; 2-layer model 51-52

item response theory (IRT) 153-156 iterative refinement 29, 62-63, 129 incorrect options and correct options 10, 24-25, 83, 87-92, 96-97, 127- 129, 137, 155, 159, 182-184

key concept 37, 45, 125 knowledge domain, domain-specific 8, 10, 27, 42, 69, 82, 219 Korean medical example 195-200 Korean multilingual example 195-200

language processing 57-60; technique 57-58; tool 59

language, source and target 191-193 large banks, items 4-5, 28, 120, 216-17

learning outcomes 26, 149

mental representation 23, 26 misconceptions 84-85, 89, 126-127, 138, 153-155; plausible 85, 87,

89, 91, 127, 138

multilingual AIG 186-204; construct- equivalent items 191; example Korean AIG 195-200; OECD 186; statistical machine translation 218; successive item development

190-191

multiple-choice items 15, 43, 48, 83, 136-137, 216

neural network 59-60 neural question generation 59-60 n-layer modeling 49-54 non-template AIG approaches 57-58, 67

non-verbal reasoning items 160-161

operational items, exams 13, 46, 62, 71, 130-131, 133, 147, 201, 207, 212

organization, testing x, 84, 131-133, 141, 147, 173, 187, 206, 216-217, 220-223

paper-based tests, testing 1,4, 175 parent items 27, 37, 43-46, 49, 101, 109-110, 116, 133, 192,215; mathematics 44, 101, 177; medical 37, 45, 94, 109, 179 point-biserial correlation 134-136, 202 process, content-coding 151, 153-154 psychometric properties 83-84, 141

rating scales 201

rationale modelling 15, 87-89, 91, 126, 153-55, 1 16, 176-179, 181- 184, 212; benefits and drawbacks 184; correct option rationale 177, 179, 182; distractor rationale 87-88, 91, 116, 126, 178, 183; feedback 155

review process 120, 121, 124, 133,

173

rubric, rating 29, 129-130 rule-based models, algorithms 59, 91

selected-response items 10, 14-15,

  • 24, 43, 48, 81-85, 87, 123, 141, 152, 159, 213, 216, 221 semantic-based AIG 57, 59-61 simultaneous-language item modelling
  • 191-193

solution and rationale generation 1 76, 183-184

standards 8-9, 63, 131, 141-142,

  • 201, 210, 215; Common Core State Standards in Mathematics 215; educational and psychological testing 8, 210; item quality 63; quality 9, 131, 141-142, 201 stimulus 7-8, 210 subject matter experts (SMEs) x, xi, vii, 4-6, 9-10, 12, 24-31, 35, 42,
  • 46- 47, 61-73, 78, 81-87, 91, 97, 120-129, 131-134, 142, 148-151, 159, 173, 183-184, 188-193, 201-202, 207, 209-222

successive-language item modelling 190-192, 195

syntactic 49, 57-58, 61, 139, 211; categories 57-58, 61; relationships 58; similarity 139; structure 49

taxonomy 27, 46-48; item model

47- 48; Educational Objective, Cognitive Domain 27

template-based AIG 14, 42, 49, 61- 63, 103,207,211-212,218-219 test designs 4, 175, 186

test development process 217 test specifications 56 three-step AIC method 9, 11, 15, 31, 55-56, 61, 81,85, 93, 121, 137, 147, 160-163, 169, 173, 184, 189-190, 195, 211, 217-218

validating generated items 8,

120-122, 141-142, 200; validation table 124-128; multilingual test items 200-203

word occurrences 60, 139 word similarity 139 written-response essays see selected- response items

 
Source
< Prev   CONTENTS   Source