Hiring Bias and Validity in Federal Law Enforcement Selection

Bias in the Federal Agency Selection Process

There is ample evidence that the selection procedures used throughout the hiring process at the federal government agency favor white males, despite their lower scores on testing and cognitive skills relative to women. Pass rates on the interviews show a clear bias toward men across all represented groups relative to women, who rank last in this category despite having the highest test scores of any group. This pattern indicates a clear bias toward males in the interview phase of the selection process.

In addition to this bias toward males, the test is ineffective in measuring Black candidates' ability to perform their roles, given their low scores. The absence of Asian or Native American applicants in the sample also suggests that the entire selection methodology may be inadvertently screening out these groups as well. Taken together, the test and interview components reveal how skewed the selection content is toward white candidates in general and male candidates specifically, and they indicate that significant modification of the entire hiring process is required. As Lawshe (1983) argues, fairness in employee hiring must be predicated on methodologies and practices that provide each applicant an equitable opportunity for testing, skills validation, and prediction of future performance.

The agency must undertake a thorough audit of its selection process, focusing specifically on the interview and written test phases. Because the entire methodology was created before the Uniform Guidelines on Employee Selection Procedures were established in 1978, many of its core concepts and methods are outdated and do not reflect current best practices. To address the exclusionary nature of the process — evident from the summary of interview and test results — the agency must re-evaluate its core assumptions and elements pertaining to knowledge, skills, abilities, and other characteristics (KSAOs). It must also redefine how competency assessments are conducted. On this latter point, competency-based assessments must be modified to reflect the changed nature of the position, with the explicit recognition that role requirements may differ substantially from those in place when the job was originally defined (Hagan, Konopaske, Bernardin, & Tyler, 2006). While interim solutions exist, the most thorough approach is to design an entirely new selection process through a comprehensive re-evaluation and re-engineering of each stage of recruitment, training, and retention.

The individual components of a selection process must be assessed periodically to ensure they remain in compliance, even when the overall process does not appear to produce adverse impact. The most significant reason for this added precaution is that individual components may be gradually moving out of compliance, or their negative effects may be neutralized by other elements of the broader process. One component — such as the written test in the government agency example — may so severely discriminate against Black candidates that its errors are obscured in the aggregate by other components.

Why Individual Components Must Be Assessed Separately

Previous studies of content- and criterion-based validity tests also demonstrate how important periodic review of each selection component is for ensuring long-term validity and the statistical independence of longitudinal results (Arthur, Bell, Villado, & Doverspike, 2006). Within this case, the two areas of the written test require continual evaluation to ensure that position screening remains aligned with current job requirements and that the overall selection process maintains fidelity and uniformity of focus.

Content-related validity measurements are highly effective for measuring the degree to which a test reflects domain expertise and knowledge (Carrier, Dalessio, & Brown, 1990). However, this approach is not effective for measuring the probability of success within a given selection process when there is no clear relationship between the sampled items and a well-defined domain. Content validity is best applied in stable, well-researched fields where the selection process draws on a broad, well-understood body of knowledge (Carrier, Dalessio, & Brown, 1990).

Content Validity vs. Criterion-Related Validity

Criterion-related validity studies are most effective when the selection process is defined through a series of predetermined metrics and performance indices that can be extrapolated across the full population of job applicants and employees (Arthur, Bell, Villado, & Doverspike, 2006). The use of such indices to measure the probability of success in a given role is highly dependent on situational context, the role of longitudinal data in measuring performance, and the overall predictability of the criterion-related validity approach (Carrier, Dalessio, & Brown, 1990). Criterion-related validity scores must also be continually evaluated for applicability to a given position, ensuring that the evolving nature of the role and its selection process remain aligned (Hagan, Konopaske, Bernardin, & Tyler, 2006).

For the government agency hiring entry-level law enforcement personnel, the criterion-related validity approach is the stronger choice. It offers a greater chance of reducing variability attributable to race, gender, and age. Moreover, criterion-related validity studies can be analyzed statistically over time to assess how effective the selection process has been, providing an ongoing mechanism for improvement.