Menu
Home
Log in / Register
 
Home arrow Psychology arrow The Wiley Blackwell handbook of the psychology of recruitment, selection and employee retention
Source

Validity of GMA for predicting overall and task performance and training

To date, numerous meta-analyses on the validity of GMA and cognitive abilities for predicting job performance and training proficiency have been undertaken. Meta-analyses have been performed in many countries and regions, including the United States, European Union (EU), Japan and South Korea. This chapter includes a summary of the major metaanalytic findings in different countries and for various organizational criteria. These data are from major sources, including Ghiselli (1966, 1973), Hunter (1983, 1986; Hunter & Hunter, 1984), Hartigan and Wigdor (1989), Schmitt and colleagues (1984), Levine and colleagues (1996), Salgado and colleagues (2003a, 2003b), Hulsheger and colleagues (2007) and Lang and colleagues (2010), among others. Additional meta-analyses were reviewed in Ones, Diclhert and Viswesvaran (2012).

The seminal quantitative syntheses of the criterion-related validity of cognitive ability tests must be attributed to Ghiselli (1966, 1973), although they were not a proper metaanalysis. Ghiselli (1966, 1973) grouped validity coefficients from thousands of studies and found an average observed validity of 0.25 for predicting job performance ratings. Ghiselli’s data were subsequently re-analyzed by Hunter and Hunter (1984).

Probably the most comprehensive meta-analyses were conducted by Hunter (1983, 1986; Hunter & Hunter, 1984), with a database consisting of 515 studies (N = 38,620), carried out using the GATB database of the US Employment Service (USES). Hunter presented the results for two criteria: job performance ratings and training success. He corrected the observed mean validity for criterion unreliability and range restriction. Hunter used 0.60 as an estimate of criterion reliability for a single rater, 0.80 as training reliability and 0.67 as the ratio between the standard deviation of the selected group and the standard deviation of the large group. Hunter found an average operational validity of 0.45 for job performance ratings and 0.54 for training success (see Table 7.1). Hunter and Hunter’s (1984) work was subsequently replicated by the US National Research Council (Hartigan & Wigdor, 1989), but with three main differences with respect to Hunter and Hunter’s meta-analysis: the number of studies was enlarged by 264 (N = 38,521), the estimate of job performance ratings reliability was 0.80 and they did not correct for range restriction. Under these conditions, the panel found an es timated average operational validity of 0.22 (K= 755; N = 77,141) for predicting job performance ratings. Interestingly, the analysis of the 264 additional studies showed an average observed validity of 0.20. Several studies have shown that Hunter and Hunter’s estimate of job performance ratings reliability was very accurate with interrater reliability for a single rater of 0.52 (Rothstein, 1990; Salgado et al., 2003a; Salgado & Moscoso, 1996; Salgado & Tauriz, 2014; Viswesvaran, Ones & Schmidt, 1996). If Hunter and Hunter’s figures are applied to the mean validity found by the panel, average operational validity are 0.38, a figure closer to Hunter and Hunter’s result for job performance ratings.

Another meta-analysis was carried out by Schmitt, Gooding, Noe and Kirsch (1984), using the studies published between 1964 and 1982 in the Journal of Applied Psychology and Personnel Psychology. They found an average validity of 0.22 (uncorrected) for predicting job performance ratings. Correcting this last value using Hunter and Hunter’s figures for criterion unreliability and the ratio of range restriction, the average operational validity was essentially the same in both studies (see Hunter & Hirsh, 1987).

Meta-analyses of the criterion-related validity of cognitive ability have also been performed for specific jobs. Pearlman, Schmidt and Hunter (1980) found that GMA was a valid predictor of job performance for clerical occupations, and its validity generalized across job families and organizations. Schmidt, Hunter and Caplan (1981) meta-analysed

Table 7.1 Average validity of general cognitive ability for predicting job performance and training in different countries.

Country

K

N

P

90CV

Job Performance

USAa

425

32,124

0.45

0.29

European Unionb

93

9,554

0.62

0.37

France

26

1,445

0.64

0.48

Germany

8

701

0.68

0.35

Belgium & The Netherlands

15

1,075

0.63

0.20

Spain

11

1,182

0.64

0.45

United Kingdom

68

7,725

0.56

0.46

South Koreac

8

1,098

0.57

0.57

Japanc

126

26,095

0.20

0.04

Training

USAa

90

6,496

0.54

0.32

European Unionb

97

16,065

0.54

0.29

France

22

5,796

0.38

0.26

Germany

26

4,645

0.63

0.42

Belgium & The Netherlands

8

706

0.65

0.48

Spain

7

712

0.61

0.41

United Kingdom

59

11,218

0.50

0.29

Source: a = Hunter (1986, p. 344) and Hunter & Hunter (1984, p. 81); b = Salgado et al. (2003); c = Oh, Schmidt, Shaffer & Le (2008; average of two meta-analyses); Oh, Schmidt, Shaffer & Le (2008; average of three meta-analyses).

the validities for craft jobs in the petroleum industry, and Hirsh, Northrop and Schmidt (1986) summarized the validity findings for police officers. Hunter (1986), in his review of studies conducted in the US military, estimated GMA validity was 0.63. Another relevant meta-analysis for craft jobs in the utility industry (e.g., electrical assembly, telephone technicians and mechanical jobs) was carried out by Levine, Spector, Menon, Narayanon and Canon-Bowers (1996), who found an average observed validity of 0.25 and an average operational validity of 0.43 for job performance ratings. For training success, the average observed validity was 0.38 and the average operational validity was 0.67. Applying Hunter and Hunter’s estimates for criteria reliability and range restriction, the results show an operational validity of 0.47 for job performance ratings and 0.62 for training success. Both results agree with Hunter and Hunter’s findings.

Two single studies conducted in the US using large samples deserve a mention. In 1990 the results of Project A, a research project carried out in the US Army, were published. McHenry, Hough, Toquam, Hanson and Ashworth (1990) reported validities of 0.63 and 0.65 for predicting ratings of core technical proficiency and general soldering proficiency. The second large-sample study was carried out by Ree and Earles (1991), who showed that a composite of GMA predicted training performance, with a corrected validity of 0.76.

In the EU, Salgado, Anderson and colleagues conducted a series of meta-analyses using primary validity studies carried out in several European countries, including Belgium, France, Germany, Spain, The Netherlands, and the UK (Bertua, Anderson & Salgado, 2005; Salgado & Anderson, 2002, 2003a, 2003b; Salgado et al., 2003a, 2003b). They found that GMA predicted both job performance and training proficiency, and generalized validity across studies and countries. Globally, they found an operational validity of 0.62 for predicting job performance and 0.54 for predicting training success.

They also found that GMA predicted both criteria in 10 occupational families: engineering, chemistry, managerial, typing and filing, information, sales, police, mechanics, electrical and driving occupations. Hulsheger and colleagues (2007) conducted another meta-analysis with German validity studies and found essentially the same results.

Japanese validity studies have been collected and analysed in three meta-analyses over the last 20 years. In 1994, Takahasi and Nishida (cited by Oh, 2010, p. 14) conducted an initial meta-analysis with 15 studies (N = 5898) and found a corrected validity of 0.28. Nimura, Imashiro and Naito (2000) conducted a second meta-analysis of 24 validity studies of the new managerial aptitude test and found an observed validity of 0.18 and a corrected validity of 0.26 (N = 4420). In 2010, Oh (2010) published the third meta-analysis with 65 studies (N = 14,777), and found a corrected validity of 0.15. The aggregation of these three validity studies resulted in an average corrected validity of 0.20, which is remarkably lower than the validity size found in US and European validity studies. Nimu- ra and colleagues (2000) speculated that, in Japan, organizational citizenship behaviours (OCB) are more explicitly expected and appraised, and that GMA is a better predictor of task performance than OCB. An alternative explanation is that the values of range restriction and criterion reliability were remarkably larger than those found in American and European validity studies, and that training proficiency and job performance ratings were collapsed as a single criterion in the initial meta-analyses.

Two independent meta-analyses were conducted with Korean validity studies of GMA. Lee (2005) found a corrected validity of 0.59 (N = 665), and Oh (2010) found a corrected validity of 0.53 (N = 443). The weighted-sample average validity of these meta-analyses was 0.56 (N = 1098), which was a value similar to that found in the American and European meta-analyses.

An important finding in both American and European meta-analyses was that job complexity was a very significant moderator of GMA criterion validity. In the US Hunter and Hunter (1984) found that GMA validity dropped from 0.56 to 0.40 as job complexity decreased. The same pattern was found in the European studies, as GMA validity dropped from 0.64 for high complexity to 0.51 for low complexity. Tables 7.1 and 7.2 summarize the meta-analytic findings for the different countries and the moderator effects ofjob complexity.

 
Source
Found a mistake? Please highlight the word and press Shift + Enter  
< Prev   CONTENTS   Next >
 
Subjects
Accounting
Business & Finance
Communication
Computer Science
Economics
Education
Engineering
Environment
Geography
Health
History
Language & Literature
Law
Management
Marketing
Mathematics
Political science
Philosophy
Psychology
Religion
Sociology
Travel