Extended Time and Construct Representation

Time limits during tests are often considered to be a source of construct-irrelevant variance (e.g., Lu & Sireci, 2007), because some of the variance in scores will reflect variability in the proportion of the test that different examinees have an opportunity to reach. In some situations, then, extended time accommodations reduce that construct-irrelevant variance by allowing additional time for examinees who need it and ensuring that all examinees have the opportunity to complete the test. Of course, an assumption lurking behind that analysis is that the speed with which examinees complete the test is entirely irrelevant to their skill level. Psychometricians and disability advocates often make this assumption after noting obvious exceptions where speed is part of the intended construct to be measured (e.g., a typing test).

But is the importance of speed really limited to such unusual tests? Or could variability in examinees’ test-taking speed actually be a source of construct-re/event variance? To put this in a legal context, when is the extension of a time limit a “fundamental alteration” to a test that is not a reasonable accommodation under the Americans with Disabilities Act (Weber, 2010)?

Evidence relevant to answering these questions is spread out widely across different areas of research. Before reviewing that evidence, two red herrings are worth noting; they are irrelevant research findings that have led some scholars (e.g., Jolly-Ryan, 2007; Kelman & Lester, 1997) to doubt the value of speed. First, on any particular test taken by a group of examinees, there often is no substantial linear relationship between how long an examinee takes to complete the test and the accuracy of his or her answers (e.g., Lovett, Lewandowski, & Potts, 2017). However, this research finding only implies that there are roughly as many slow-and-accurate examinees as there are fast- and-accurate examinees. The finding leaves unanswered the question of whether test users would want to rank the fast-and-accurate examinee higher (in skill, competence, etc.) than the slow-but- accurate examinee. A second red herring is that a given examinee will show a greater tendency toward making errors when given a tighter time limit, a phenomenon known as the speed-accuracy tradeoff (e.g., Heitz, 2014). However, examinees still van' in how much time they need to achieve a given level of performance (accuracy), again raising the question of whether it is valuable to be accurate under a tight time limit versus only under a generous one. These two common findings, then, raise more questions than they answer about the relevance of speed with respect to skill level.