Selection of Air Force Pilot Candidates: A Case Study on the Predictive Accuracy of Discriminant Analysis, Logistic Regression, and Four Neural Network Types

Abstract

We evaluated the predictive classification accuracy of discriminant analysis, logistic regression and four neural network typologies (multiple layer perceptrons, radial basis networks, probabilistic neural networks, and linear neural networks) on a flight screening program with a pass–fail criterion using several psychometric tests as predictors. A stepwise (for logistic regression and discriminant analysis) and sensitivity (for neural networks) selection procedure identified spatial visualization, eye–hand–foot coordination, and concentration capacity as significant predictors. Performance on the first few flights of the screening program was also retained as a significant predictor of final score. Regarding the accuracy of predictions, logistic regression showed the highest accuracy (77%), with high sensitivity (92%) but low specificity (31%). Discriminant analysis had high sensitivity (77%) and high specificity (64%). However, it had the second lowest accuracy (74%). The best performing neural network type was the multiple layer perception, which showed high sensitivity (85%), the second highest specificity (47%), and high accuracy (76%). Radial basis networks and probabilistic networks both fail to predict correctly the candidates who fail on the flight screening program (0% specificity).