A Note on a Commonly Used Ridge Regression Monte Carlo Design

Abstract

Ridge estimators are usually examined through Monte Carlo simulations since their properties are difficult to obtain analytically. In this paper we argue that a simulation design commonly used in the literature will give biased results of Monte Carlo simulations in favor of ridge regression over ordinary least square estimators. Specifically, it is argued that the properties of ridge estimators that are functions of p distinct regressor eigenvalues should not be evaluated through Monte Carlo designs using only two distinct eigenvalues.

Keywords:

• Design matrix
• Excess mean square error
• Monte Carlo simulation
• Multicollinearity
• Ridge regression

Notes

The regressors of (1.1) are formally assumed to be fixed (non-random) but it is well known that a considerable improvement in simulation precision may be obtained by drawing a separate sample in each replicate (Edgerton, 1996) and therefore Monte Carlo simulations usually involve fixed regressors in repeated sampling. In this sense E[X′X] represents the Monte Carlo average of the fixed X′X matrix.