Improving the Pilot Selection System: Statistical Approaches and Selection Processes

Abstract

Pilot selection systems traditionally use one of three statistical approaches to model candidate performance: multiple linear regression, linear discriminant analysis, and logistic regression. This article reviews the literature comparing selection decisions using these three approaches and compares the classification accuracy of linear discriminant analysis and logistic regression to the results from two Monte Carlo simulations. Methods for adjusting to a pilot shortage are described for each statistical approach. In the second half of the article, we describe a selection system using a progressive process, rather than the traditional single- or multistage process. We discuss how system operators can adjust each of the processes to deal with a pilot shortage.

Notes

1. ¹The area under the ROC curve is a nonparametric measure of sensitivity; that is, how well the decision maker can make correct judgments. Under a two-alternative, forced-choice paradigm, the area under the ROC curve is equal to the percentage of correct choices (Green & Swets, 1974). The classification task described by Maroco et al. can be conceptualized as a two-alternative, forced-choice task. Thus, the area under the curve can be thought of as the percentage of correct classifications.

2. ²Throughout their article, Lei and Koehly (2003) referred to the group of primary interest as the “larger group.” This terminology might allow outcomes involving the larger training sample size group to be confused with outcomes pertaining to the GOI. We have changed the terminology of the larger and smaller groups to the group of interest and secondary group to reduce possible confusion.

3. ³One potential response to the impending pilot shortage is increased training to reduce the percentage of training failures to less than 5%. In situations where the failure rate is extremely low, MLR might be the only reliable statistical approach (Harrison, 2002). To use MLR in this situation, the organization must systematically record measures of trainee quality and progress for all of the successful trainees.

4. ⁴The trade-off between the false positive error rate and the false negative error rate caused by moving a cut score is well known in industrial/organizational psychology. For an early discussion of this trade-off, see Blum and Naylor (1968).