FOUR. Statistical Analysis of the Selection Process

In this chapter, we report findings from our statistical analysis of the firefighter selection process, using the 2013 cohort of applicants. The objective of this analysis was twofold: (1) to identify steps in the process where a large proportion of the applicants were eliminated and (2) to identify steps in the process that affected the racial, ethnic, and gender representation of the applicant pool. The former can assist in identifying the steps most useful for narrowing the applicant pool, and the latter can inform which elements in the process are most likely to affect diversity. We considered these findings in formulating our recommendations to improve efficiency, minimize potential disparate impact on diversity, and improve the overall fairness and transparency of the firefighter hiring process.

Data

We obtained data from the Personnel Department on all firefighter candidates screened in the 2013 hiring process. Before analyzing the selection process, we reviewed the applicant data files for discrepancies and duplicate information. In a small number of cases, we eliminated applicant records because they appeared in the file more than once, and we adjusted applicant records that were implausible (e.g., when there was evidence of not passing an early step followed by evidence of passing all subsequent steps). Therefore, our numbers approximate but are not identical to those published by the Personnel Department. The full selection process includes a number of intermediary steps–including filling out paperwork, scheduling, and showing up in person–that are part of the ten steps outlined by the Personnel Department (as described in the previous chapter). However, for the results presented in this chapter, we are concerned only with those steps that involve an explicit decision point (i.e., continue on in the process or reject). As such, in this chapter, some intermediary steps are combined and reported as a single decision point.

We also obtained summary statistics on CPAT pass rates from the CPAT testing office in Orange County, California, to estimate what the pass rates would have looked like for the CPAT step had the pass rates not been restricted by the “first come, first served” approach for narrowing the applicant pool.