Validity Considerations in Test Design
Test design should begin with plans to gather the evidence needed to support the validity of intended interpretations of test scores for specified uses. Sometimes referred to as a validity argument, Standard 1.2 states the following:
A rationale should be presented for each intended interpretation of test scores for a given use, together with a summary of the evidence and theory bearing on the intended interpretation.
The value of conducting an overall review of the evidence supporting the validity argument by independent experts is stated in Standard 4.6.
When appropriate to documenting the validity of a test for intended uses, relevant experts external to the testing program should review the test specifications to evaluate their appropriateness for intended uses of the test scores and fairness for intended test takers. The purpose of the review, the process by which the review is conducted, and the results of the review should be documented. The qualifications, relevant experiences, and demographic characteristics of expert judges should also be documented.
The first chapter of the 2014 Standards describes different types of validity evidence needed to support specific types of interpretations and uses, including alignment of content and cognitive processes to support interpretations of mastery of specific knowledge or skills, correlational analyses to support predictive interpretations, and analyses of internal and external relationships to support interpretations of the meaning of the construct being measured.