Robust Linear Model Selection by Cross-Validation

Abstract

This article gives a robust technique for model selection in regression models, an important aspect of any data analysis involving regression. There is a danger that outliers will have an undue influence on the model chosen and distort any subsequent analysis. We provide a robust algorithm for model selection using Shao's cross-validation methods for choice of variables as a starting point. Because Shao's techniques are based on least squares, they are sensitive to outliers. We develop our robust procedure using the same ideas of cross-validation as Shao but using estimators that are optimal bounded influence for prediction. We demonstrate the effectiveness of our robust procedure in providing protection against outliers both in a simulation study and in a real example. We contrast the results with those obtained by Shao's method, demonstrating a substantial improvement in choosing the correct model in the presence of outliers with little loss of efficiency at the normal model.

Key Words:

• Bounded influence
• Construction sample
• Outliers
• Prediction error
• Robust prediction
• Validation sample