Determining the optimal ridge parameter in logistic regression

Abstract

A closed interval based on the eigenvalues of the explanatory variables in the dataset is analytically derived to contain the ridge parameter that minimizes the mean squared error (MSE) of the coefficient estimators in a logistic regression model. After specifying the required accuracy, a Fibonacci search can efficiently locate the optimal ridge parameter within such a closed interval. Based on a simulation comprising 2,000 replications of three sample sizes (100, 200, and 1,000) from a logistic regression model consisting of two correlated variables with correlation coefficients of 0.90, 0.95, and 0.99, and one independent variable, it is confirmed that, using the true mean squared error criterion, the relative efficiency of the estimator with the optimal ridge parameter is clearly higher than those of estimators using six commonly used ridge estimators. Finally, using a real-life data set of small size and changing the criterion to the asymptotic mean squared error, comparisons with the same six estimators show that the relative efficiency of the estimator with the optimal ridge parameter is better than or equal to others.

Keywords:

• Bounds of the ridge parameter
• Efficiency
• Logistic regression
• Multicollinearity
• Simulation

MATHEMATICAL SUBJECT CLASSIFICATION:

• 62J07
• 62J02