# Correct Option With Rationale

The second method includes the solution, as well as a rationale for the solution. Hence it is called the correct option with rationale method. When the logical structures model is used for generation, examinees are presented with their answers, as well as the correct answer and values from the item in the formula, algorithm, rule, and/or procedure that were used to produce the correct answer (see the middle example in "Method" panel in Table 10.2). The feedback is presented as follows: "You selected [Option]. The correct option is [Correct Option] because it is [I2] to [11] + [I2] + [13]."

When the key features model is used, two different versions are available. The first version is based on a list of all key features that were used in the stem of the item model. Hence this version can be described as the

Table 10.3 Key Features Medical Item Model

 Parent Item: A 22-year-old female sees her doctor and reports that she's been experiencing a mild cough and slight body aches that have developed over a few days. Upon examination, she presents with an oral temperature of 37°C. What is the most likely diagnosis? 1: Hay Fever 2: Seasonal Flu 3: Otitis Media 4: Common Cold 5: Streptococcal Infection Item Model: Stem A [Age]-year-old female sees her doctor and reports that she's been experiencing a [Cough Type] cough and [Body Aches] that have developed [Onset], Upon examination, she presents with an oral temperature of [Temperature], What is the most likely diagnosis? Element Age: 18 to 30, by 1 Cough Type: (1) mild, (2) hacking, (3) severe Type of Body Aches: (1) slight body aches, (2) slight body pains, (3) severe body aches, (4) severe body pains Onset of Symptoms: (1) over a few days, (2) within 3-6 hours, (3) suddenly Fever Temperature: (1) 37°C, (2) 37.8°C, (3) 39°C, (4) 39.5°C Key Correct Option: Common Cold, Seasonal Flu Distractor Bronchitis Streptococcal Infection Hay Fever Otitis Media Acute Sinusitis Bacterial Pneumonia

correct option with rationale—key features list. This rationale includes all of the key features (see the middle example labelled 2a in the "Method" panel in Table 10.4). In our example, the key features include the correct option, age, cough type, type of body aches, onset of symptoms, and fever temperature. The text for this rationale is structured as a succinct list for the examinee that is focused on all of the key features in the

Table 10.4 Three Methods for Providing Feedback Using Key Features Medical Example

 Method Method 1 (Correct Option): You selected (Option]. The correct option is [Correct Option], Method 2a (Correct Option With Rationale—Key Features List): [Correct Option) is the most correct option for this[Age]-year old patient who has a [Cough Type] cough, [Type of Body Aches], with symptoms appearing [Onset of Symptoms] and temperature of [Fever Temperature], These are the key features of [Correct Option], Method 2b (Correct Option With Rationale—Key Features Set): [Correct Option] is the most correct option, as the patient has a temperature of [Fever Temperature]. The key features of this diagnosis include [Key Feature], Method 3 (Correct Option and Distractor Rationale): [Correct Option] is the most correct option for this [Age]-year old patient who has a [Cough Type] cough, [Type of Body Aches], with symptoms appearing lOnset of Symptoms] and temperature of [Fever Temperature], The option you selected is not correct because [Distractor Rationale], Key Feature 1. [Cough Type] cough and [Type of Body Aches], appearing [Onset of Symptoms] Dis tractor Rationale General: 1. the presented symptoms are not usually associated with the Common Cold. 2. the presented symptoms are not usually associated with the Seasonal Flu. For Correct Option Common Cold: 3. the Seasonal Flu has the symptoms severe cough, severe body aches, bedridden, and fever. 4. Bronchitis has the symptoms severe cough, mild body aches, bedridden, and fever. 5. Streptococcal Infection has the symptoms mild cough, mild body aches, bedridden, and fever. 6. Hay Fever has the symptom mild cough. 7. Otitis Media has the symptom fever. 8. Acute Sinusitis has the symptoms mild cough and fever. For Correct Option Seasonal Flu: 1. Common Cold has the has the symptoms mild cough, mild body aches, and fever. 2. Bronchitis has the symptoms severe cough, mild body aches, bedridden, and fever. 3. Streptococcal Infection has the symptoms mild cough, mild body aches, bedridden, and fever. 4. Hay Fever has the symptom mild cough. 5. Otitis Media has the symptom fever. 6. Bacterial Pneumonia has the symptoms severe cough, severe body aches, bedridden, and fever.

problem. It could be presented as follows: "[Correct Option] is the most correct option for this [Age]-year old patient who has a [Cough Type] cough, [Body Aches], with symptoms appearing [Onset] and temperature of [Temperature], These are the key features of [Correct Option].''

The second version is based on a set of the key features in the stem of the item model. As a result, this version is called the correct option with rationale—key features set (see the middle example labelled 2b in the "Method" panel in Table 10.4). The text for this rationale may contain some of the elements from the key features list. For example, the text for a key features set could begin with "[Correct Option] is the most correct option, as the patient has a temperature of [Temperature]", where the key features include correct option and fever temperature. But the text also contains at least one new element that includes a set of the key features. Hence a new variable called "key feature" (see the second panel in Table 10.4) must be created for the item model. The text for the new key features element contains content from the cough type, type of body aches, and onset of symptoms features. This is a more complex type of rationale because it contains existing key features from the model used to generate the items. But it also contains new elements that combine some of the existing key features from the model used to generate the items. The benefit of this type of rationale over a more simplistic list structure is that it permits feedback designed to draw attention to specific combinations of key features. These combinations, in turn, may yield a more complete explanation of how to solve the problem. In our example, the key feature of temperature is present in the first sentence of the rationale. The intended effect of this rationale structure is that the examinee is immediately drawn to the patient's temperature. Any key feature in the cognitive model can be used to highlight or emphasize a specific element required in the rationale explanation. Then in the second sentence of the rationale, the remaining key features of cough type, type of body aches, and onset of symptoms are combined into a single element and presented to the examinee as part of the explanation. Again, any combination of the key features can be included or created to support the explanation in the rationale. For example, the key features can be organized by the sources of information in the cognitive model. In this case, the physical examination element (i.e., fever temperature) is presented first in the rationale, followed by the history elements (cough type, body aches, onset). The text in the rationale is presented as follows: "The key features of this diagnosis include [Key Feature]". Because many different combinations of key features are permissible (i.e., [Cough Type] cough and [Type of Body Aches], appearing [Onset of Symptoms]), many rationales can be generated. For this reason, the correct option with rationale—key features set method is very flexible. It is also adaptive to the instructional focus that is intended to be emphasized as part of the feedback provided to the examinees.

# Correct Option With Distractor Rationale

The third method is the most comprehensive because it includes the solution, as well as a rationale for the correct and incorrect options. It is called the correct option with distractor rationale method. When the logical structures model is used, examinees are presented with the correct answer and values from the item in the formula, algorithm, rule, and/or procedure that were used to produce the correct answer. They are also presented with the values in the formula, algorithm, rule, and/or procedure that were used to produce the incorrect answer they selected (see the bottom of the "Method" panel in Table 10.2). This is the most complex type of rationale because it provides an explanation for both the correct option and incorrect options in each generated item. It is only used when examinees select an incorrect option. The feedback is presented as follows: "The correct option is [Correct Option] because it is [I2] to [11] + [I2] + [13]. The option you selected is not correct because [Distractor Rationale]." The distractor rationales for our 1 -layer mathematics example are provided in the "Distractor Rationale" panel in Table 10.2. Recall that the distractors were created by SMEs using common errors and misconceptions initially, as described in Chapter 5.

When the key features model is used, examinees are presented with text that may contain elements from the key features list or from the key features set, but it always contains elements that provide an explanation for why the distractor selected by the examinee yields the incorrect solution. The example presented in Table 10.4 states, "[Correct Option] is the most correct option for this[Age]-year old patient who has a [Cough Type] cough, [Type of Body Aches], with symptoms appearing [Onset of Symptoms] and temperature of [Fever Temperature]. The option you selected is not correct because [Distractor Rationale]." The distractor rationales are provided in the "Distractor Rationale" panel in Table 10.4. Creating distractor rationales draws on logic that is comparable to the approach that was used for systematic distractor generation. The common elements among the distractors that produce an incorrect option are presented to the examinees as the explanation for why the distractors would be erroneous responses. To permit these associations among the elements, each distractor rationale is coded so that it will appear whenever a set of distractors with the same features are used as the incorrect options for the generated items. For example, the incorrect options of bronchitis, streptococcal infection, hay fever, otitis media, and acute sinusitis are most similar to the common cold. Therefore, when these options are used as incorrect options for a generated item, the distractor rationale can state, "The option you selected is not correct because [Distractor Rationale]." The rationale can be a general or a specific summary of the symptoms. The third type of rationale provides examinees with the most comprehensive feedback because one explanation is included for the correct option and another explanation is included for the incorrect option selected by the examinee.