Principal component regression under exchangeability

Abstract

A Bayesian approach to principal component regression is formulated. Under exchangeability of regression coefficients a Bayes principal component regression estimate is defined which can be viewed as a class representation for several known estimates. An information statistic is developed which measures gain of information about the parameters when a component is used as a regressor. This measure serves as a criterion for selecting informative components to estimate the coefficients. The information statistic coherently combines two widely practiced selection criteria, the variance of a component and the correlation between a component and the dependent variable, into a single criterion. Although the proposed measure is developed in a Bayesian framework, it is useful and can be easily applied in the classical setups. An example is analyzed to illustrate the application of the information selection statistic and the new Bayes estimate.

Keywords:

• Bayes estimate
• entropy
• information statistic
• random coefficients
• standard regression model