On PRESS and deletion bootstraps in linear regression

Abstract

Bootstrap technique has an extensive use in almost all branches of statistics including regression analysis. The classical regression theory heavily depends on normality assumption of errors, which does not often hold in practice. Bootstrap technique can be readily applied to a situation when we do not assume any distribution for the error and that is why bootstrapping in regression is becoming very popular. In reality the true errors are not known, in that case we need a proper substitute of the residuals. The OLS residuals are being used as substitutes of errors from the very beginning but it is now evident that they may break down easily in the presence of unusual observations. In our bootstrap study we consider some other types of residuals such as PRESS and observed that it may produce results even worse than the OLS in the presence of outliers. Then we have considered deletion residuals based on robust fits. The usefulness and/or limitations of bootstrap techniques based on different types of residuals such as PRESS and deletion residuals are studied through some real life examples and a Monte Carlo simulation experiments under a variety of situations.

Keywords:

• Fixed-X bootstrap
• Group deletion
• Masking
• Monte Carlo simulation
• Robust regression

AMS SUBJECT CLASSIFICATION:

• 62H11