Applicant experience and prior knowledge
Many simulations assume that applicants already have the experience, knowledge, skills and abilities required for performance. As a result, simulations may not be appropriate for entry-level jobs or more generalist positions, especially if post-hire training is to be provided (U.S. Merit Systems Protection Board, 2009; Whetzel et al., 2012).
While the use of technology has reduced administration costs in some cases, simulations can be more expensive to build and maintain than other assessment types. Because of their specificity, they require significant input from subject matter experts and test development professionals. When not delivered online, the equipment required may be expensive, as are the costs of assessment proctoring. In addition, as jobs change, simulations will need to be revised more frequently than other assessment types. This is of particular concern given the rapidly changing nature of work.
Costs are also directly related to the level of fidelity desired. Increasing fidelity means increasing costs, though the cost of increasing fidelity across different simulation types is not equal (Walsh & Jaye, 2012). For example, low-fidelity simulations can leverage automated scoring, while higher-fidelity simulations often rely on expensive human rating (Boyce et al., 2013). Similarly, the use of multimedia in a low-fidelity simulation such as a branching role-play may increase development costs compared to their in-person counterparts, but may be more cost-effective to administer over time when the resources required to administer and score an in-person assessment are taken into consideration. When considering the use of simulations, practitioners are advised to examine the range of cost factors and consider utility. This is particularly true given the research on fidelity and validity in selection contexts is not definitive. Taking an example from an educational context, Lapkin and Levett-Jones (2011) used a cost-utility analysis and found medium-fidelity simulations for nurse education were more cost-effective than the high-fidelity simulations, as the medium-fidelity simulation was one-fifth the cost and resulted in the same level of knowledge acquired and same level of student satisfaction.