Methods for Assessing Personal Attributes and Noncognitive Skills

The foregoing review of noncognitive or background frameworks can be seen as a compilation of various constructs one might wish to measure for various purposes. Constructs can be measured in a variety of ways, with some constructs more amenable to some approaches and others to other approaches. Figure 10.1 presents a constructs-by-methods taxonomy illustrating this concept.

The constructs are chosen for illustrative purposes, but follow the 21st-century skills taxonomy developed in the NAS/NRC framework (Pellegrino & Hilton, 2012). That framework organizes constructs into cognitive, interpersonal and intrapersonal categories, and shows crosswalks with many

Likert Scale (BARS, AVs)

Forced Choice

i

Biodata

D

are

Personal Statement

Likert Scale Forced Choice Letter of Recommendation

are

Situational Judgment Tests

О -к

  • s g- 8
  • -sos « ? g,

f? ТЭ Й

^ S 5

I* | -E s ^ 1

^ -S >3 g >

H

Face to face -- Behavioral

в1

Behavioral video interview Collaboration/Role Play

Multiple Choice Short Answer, Essay Interactive (game)

Cognitive

Problem Solving, Critical Think. Idea Production, Creativity Information Technology Scientific Inquiry Skills

Global/Cultural

Skills

©

©

о

©

©

©

-

О

©

-

о

©

о

о

©

-

-

-

-

©

о

-

-

-

-

о

-

О

О

-

©

©

©

о

Inter- personal

Communication

Skills-Oral

Communication

Skills-Written

Leadership Skills

Teamwork,

Collaboration Skills

0

-

©

©

©

о

0

0

©

-

©

©

-

-

-

о

0

0

©

-

©

о

©

о

©

©

©

-

©

о

©

о

о

о

Intra- personal

Self-Regulation Skills Emotional- Regulation Skills Interests & Attitudes Career Aspirations Subjective Well Being Health

-

О

о

о

-

о

о

0

©

-

©

О

0

©

-

0

-

0

-

©

— Assessments could be developed but there is no need for them (better assessments are available) • Assessments are available and widely used (mature technology)

® Assessments are being developed or in experimental usage (developing technology)

О Assessments are in early exploration stages (research status)

0 No research in this area

Figure 10.1 Taxonomy of constructs by measurement approaches.

other 21st-century skills frameworks (it also crosswalks with the Big 5, in that Agreeableness and Extraversion are mostly aligned with interpersonal skills, and Conscientiousness, Emotional Stability and Openness are mostly aligned with intrapersonal skills).

The horizontal dimension lists five categories of methods—self ratings, others’ ratings, situational judgment tests, interviews and tests. Within each method there are variants, such as Likert scale (i.e., rating scales), forced-choice, biographical data (biodata) and personal statements as methods of self-ratings.

The cell entries are indicators of the prevalence of the methodology for measuring the particular construct. The filled circle (•) indicates that the methods for measuring that construct are available and widely used; the partially filled circle (©) indicates that methods have been developed, but are not as widely used (perhaps still in a research mode); the unfilled circle (O) means that applications of the method for measuring the construct are even rarer; and the tiny circle (°) indicates that the particular method is not used to measure the construct, and there is not even any research in the area, at least that I am aware of. The dash (—)indicates that there is no need for research or that there are other better methods available for measuring the construct. Admittedly these are all subjective judgments, but perhaps they serve a purpose in providing a general sense for the lay of the construct-byassessment-method land.

Some general observations can be made about Figure 10.1. One is that rating scales (e.g., Likert scales) are quite common methods. Rating scales are a general, all-purpose and default methodology for measuring just about any noncognitive construct. As discussed in the next section, rating scales have their limitations, and some of the most innovative and important advances in the past few years, and going forward into the future, are likely to be in the development of alternatives to rating scales in measuring noncognitive constructs. A second observation is that letters of recommendation are certainly one of those alternatives to rating scales, and letters of recommendation also are commonly used to measure just about any noncognitive construct. Some other observations are that there are tests (or performance measures) to measure cognitive constructs (particularly problem solving, critical thinking, information technology, scientific and communication skills), but only early research programs to develop tests in many of the other areas (e.g., teamwork, cultural skills, creativity, self-regulation skills). There are also many constructs for which there seems to be no research regarding certain measurement approaches. But Figure 10.1 is designed to point out that in principle there could be.

I now provide more in-depth discussion of the various methods, in turn. Standard 3.0 (Standards for Fairness) provides a key principle that guided this review:

All steps in the testing process, including test design, validation, development, administration, and scoring procedures, should be designed in such a manner as to minimize construct irrelevant variance and to promote valid score interpretations for the intended use for all examinees in the intended population.

(AERA et al., 2014, p. 63)

 
Source
< Prev   CONTENTS   Source   Next >