Bootstrap confidence interval of ridge regression in linear regression model: A comparative study via a simulation study

Abstract

It is well known that the variances of the least squares estimates are large and they can be far away from their true values in the case of multicollinearity. Therefore, the ridge regression method can be used as an alternative to the least squares method. However, the ridge estimator has a disadvantage that its distribution is unknown, so only asymptotic confidence intervals are obtained. The purpose of this paper is to study the impact of several ridge regularization parameters on the mean interval lengths of the confidence intervals and coverage probabilities constructed by the ridge estimator. A bootstrap method for the selection of ridge regularization parameter is used as well as the parametric methods. In order to compare the confidence intervals, standard normal approximation, student-t approximation and bootstrap methods are used and comparison is illustrated via real data and simulation study. The simulation study shows that the bootstrap choice of ridge regularization parameter yields narrower standard normal approximated confidence intervals than the PRESS choice of ridge regularization parameter but wider standard normal approximated, student-t approximated and bootstrap confidence intervals than the GCV choice of ridge regularization parameter.

Keywords:

• Bootstrap
• ridge regression
• regularization parameter
• signal to noise ratio
• confidence intervals
• coverage probability

MATHEMATICS SUBJECT CLASSIFICATION:

• 62F40
• 62J07
• 11K45