Test Validation Plan

As stated by Messick (1993, p. 13), “Validity is a unitary concept ... [it] refers to the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of interpretations and actions based on test scores.” Although test validation is a critical component of test design, it is unfortunately not often accounted for as part of preliminary design work. We believe that a validation plan must be completed as part of the design efforts for a test. Kane (2006, this volume) provides guidance on validation strategies.

To ensure that validation is given sufficient attention early in a program’s development, it is helpful to create a plan for the types of evidence and research needed to substantiate validity using a particular framework. The framework should account for both construct-relevant and construct-irrelevant factors (AERA et al., 2014, Standard 1.25, pp. 30-31). Table 22.1 presents a sample framework used at ETS. The framework allows the program to plan for the types of research needed and document the research that is currently underway or completed. Gathering some important evidence may not occur within the first year of a new test’s use; it must be systematically and purposefully collected over the course of several years.

< Prev   CONTENTS   Source   Next >