A ridge to homogeneity for linear models

Abstract

In some heavily parameterized models, one may benefit from shifting some of parameters towards a common target. We consider $L^2$ shrinkage towards an equal parameter value that balances between unrestricted estimation (i.e. allowing full heterogeneity) and estimation under equality restriction (i.e. imposing full homogeneity). The penalty parameter of such ridge regression estimator is tuned using leave-one-out cross-validation. The reduction in predictive mean squared error tends to increase with the dimensionality of the parameter set. We illustrate the benefit of such shrinkage with a few stylized examples. We also work out an example of a heterogeneous panel model, including estimation on real data.

Keywords:

• Shrinkage
• homogeneity restrictions
• ridge regression
• predictive mean squared error
• cross-validation
• heterogeneous panel data

Acknowledgments

I thanks the Editor and two anonymous referees for useful suggestions; also, Wessel van Wieringen, Lukáš Lafférs and Daniel Henderson for valuable comments. This research was presented at the Workshop in Model Selection, Regularization, and Inference in Vienna, the 12th International Conference on Computational and Financial Econometrics in Pisa, the 5th Conference of Deutsche Arbeitsgemeinschaft Statistik in Munich, and the Czech Economic Society and Slovak Economic Association Meeting in Brno.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Note that the penalty term, apart from a multiplicative constant, can be rewritten as ${\left(||{\text{β}}_j|{|}_{j=1}^P-\overline{\text{β}}\right)}^{\mathrm{\prime }}\left(||{\text{β}}_j|{|}_{j=1}^P-\overline{\text{β}}\right)={\left({\mathrm{\Xi }}_P||{\text{β}}_j|{|}_{j=1}^P\right)}^{\mathrm{\prime }}\left({\mathrm{\Xi }}_P||{\text{β}}_j|{|}_{j=1}^P\right)={\left(||{\text{β}}_j|{|}_{j=1}^P\right)}^{\mathrm{\prime }}{\mathrm{\Xi }}_P\left(||{\text{β}}_j|{|}_{j=1}^P\right).$ Hence, this estimator can be interpreted as a generalized ridge estimator with weight matrix ${\mathrm{\Xi }}_P.$ I thank Wessel van Wieringen for this observation.

Additional information

Funding

This research was supported by the grants 17-26535S and 20-28055S from the Czech Science Foundation (Grantová Agentura Ceské Republiky) and the Access Industries professorship from the New Economic School.