Logical Constraint Coding Using Bitmasking
Masking is a technique used in computer science to read a defined subset of information. It can also be used to determine what information should be extracted (Marino, Zhou, Zhao, Wang, Wu, & Dinov, 2019). To bitmask is to represent conditions in binary form where each value can only contain two outcomes, thereby producing an outcome that is in one of two states. Bitmasks are used to interpret large amounts of information by sampling and reading only the relevant bits as defined by the mask, where each bit defines a required condition. Bitmasks are often used in data encryption where bits of information can be used to check for authenticity (Vijayanand, Devaraj, Kannapiran, & Kartheeban, 2016). Bitmasking can also be used to describe and simplify how constraints are defined when generating items.
When we generate items, two states or outcomes are possible: An item should be generated (1) or it should not be generated (0). Because this state can be defined for each pair of elements in a cognitive or item model, values in each element pair can be represented as an element constraint matrix. The dimension of this matrix is defined by the number of values each element should contain where each value is the corresponding state of whether a pair of values is permissible for generation. In our example in which two elements with three values need to be constrained, the element constraint matrix would be expressed as a three-by-three matrix where an example restriction of А1! = B3 can be coded as
In this example, each row represents a value in element A (three values in total) and each column represent a value of element В (three values in total). In a cognitive or item model that could be used for operational item generation, many elements exist, producing many separate element variable constraint matrices. These matrices, in turn, can be combined into a model constraint matrix where the dimension of the matrix is the sum count of all values for all elements. A model constraint matrix provides a single map that defines all possible permutations across all element pairs. Then, to generate the items defined in the models, bitmasks can be used to extract only the values that relate to the elements that are used where a product of that vector of values will determine whether an item can be generated. A summary of the bitmasking process using the three-element model example where an element constraint matrix between element A and В is first provided as follows:
It then updates the values into the model constraint matrix, as shown next:
Finally, item generation proceeds by evaluating whether an item can be generated using values АЗ, B5, and C3 given the specified constraints in the model. By permuting across all combinations of element values, items can be generated using this logical constraints approach. Figure 4.1 serves as a summary of the bitmasking process using the matrices from the example model in which the rows and columns of the variable constraint and model constraint matrices are labelled.
Figure 4.1 A bitmask for a model matrix defined by A(5) by B(5j by C(4)
There are important benefits of using bitmasking to describe the constraints in the cognitive and item models. With the use of binary masks, a layer of complexity for expressing the constraints is minimized. As a result, it is easier for the SME to deduce whether a value can appear with another value without having to rely on Boolean logic. Consequently, the element constraint matrix can be created in a checkbox interface that facilitates the cognitive and item modelling coding for SMEs. With the use of checkboxes, the complex nature of Boolean programming is minimized because there is no longer a need to directly translate the constraints presented in the cognitive and item models using Boolean logic. Instead, the SME describes the constraints using checkboxes. Finally, the use of bitmasking supports a more straightforward computational approach for generating test items. The use of Boolean case logic requires the creation of all possible combinations of elements. This task is memory intensive and requires long periods of computation time because all possible combinations must be evaluated before item generation can begin. But with bitmasking, each combination of values is evaluated individually, meaning that items can be generated on demand rather than first requiring that all possible combinations of elements be produced before generation can begin. In short, bit- masking is a desirable approach for generating items because it simplifies the way that SMEs define constraints, and it is computationally efficient.
Outcomes using the cold versus flu key features cognitive model (see Chapter 2, Figure 2.4) help illustrate how the elements can be manipulated to produce items. For this model, the elements and constraints define the key features that produce the correct response. If we follow the diagnosis option of the common cold (CC), we can see that this outcome is correct when age is any value between 18 and 30, cough type is mild or hacking, body aches is slight body aches or slight body pains, onset is over a few days, and temperature is 37°C or 37.8°C. In other words, the examinee must be able to recognize that any combination of the values across these five elements will produce the correct diagnosis option of CC.
The set of features that lead to the diagnosis option of CC can be contrasted with the diagnosis option of seasonal flu (SF). SF contains the same features as a CC but with different values for the elements. As a result, examinees must be able to differentiate the values for the features across these two diagnoses in order to select the correct answer. SF is the correct diagnosis when age is any value between 18 and 30, cough type is severe, body aches is severe body aches or severe body pains, onset is within three to six hours or suddenly, and temperature is 39°C or 39.5°C. The examinee's task is to recognize the differences across these key features and then to solve the problem by diagnosing the correct respiratory illness based on the configuration of the features.
The same general logic can be applied to generate items for the logical structures math cognitive model. One key difference between the two types of cognitive models is that logical structures often define a range of values. That is, logical structures cognitive models often include elements with integer values presented as a range where the minimum and maximum values are provided, along with the iterating steps. All possible values in the range are evaluated using bitmasks with each integer being either presented as a value or calculated in an algorithm during item assembly. Constraints can be used with bitmasking to prevent specific values from being used during the assembly process.