A simulation is a broad term that can refer to a number of different, more specific methods. The terms simulation, work sample and assessment centre are often confounded and the distinctions are not always clear. For example, an assessment centre can include a simulation such as a role-play or in-basket. Roth and colleagues (2008), in their meta-analysis of Black-White group mean score differences in work samples, suggest some differences between work samples and assessment centres, including ‘the types of exercises they include (exclusively simulations vs. a variety of exercises including traditional paper-and-pencil tests)’ (p. 638). Thus, this meta-analysis ofwork samples focused on simulations. Across all studies and samples they estimated a d of 0.36 for incumbents (k = 19, N = 5,611) and 0.73 for early-stage applicants (k=21, N = 2,476). In terms of incumbents, the type of sample moderated this effect size. The d for military samples was close to zero at 0.03 (k=7, N = 1,869), while non-military samples has a much larger d of 0.53 (k = 12, N = 3,742).
These effects were also moderated by exercise type and construct measured. For the early- stage applicant samples the effect sizes (corrected for measurement error in the exercises) varied considerably between exercises: in-basket exercise d = 0.82 (k = 8, N = 1,508 l, though one study with a fairly large sample size of N = 400 had ad of 1.15; after removing this study from the analysis, the corrected in-basket d was reduced to 0.60), technical exercise d = 0.78 (k = 11, N = 659), scheduling exercise d = 0.57 (k = 6, N = 201), oral briefing exercise d = 0.24 (k = 4, N = 847) and role-play exercise d= 0.24 (k = 15, N = 1,322). Roth and colleagues’ (2008) hypothesis that the oral briefing and role-play exercises would demonstrate smaller differences was supported. These authors argued that the substantial oral communication component involved in these types of exercise may account for this reduction.