Predicting Multivariate Response in Linear Regression Model

Abstract

Predicting a multivariate response vector in a linear multivariate regression model requires the estimate of the matrix of regression parameters. Stein (Stein, C. (1973). Estimation of the mean of a multivariate normal distribution. Proc. Prague Symp. Asymp. Statist. 345–381), van der Merwe and Zidek (van der Merwe, A., Zidek, J.V. (1980). Multivariate regression analysis and canonical variates. Canadian Journal of Statistics 8:27–39), Bilodeau and Kariya (Bilodeau, M., Kariya, T. (1989). Minimax estimators in the normal MANOVA model. Journal of Multivariate Analysis 28:260–270) and Konno (Konno, Y. (1990). On estimation of a matrix of mean. Unpublished manuscript; Konno, Y. (1991). On estimation of a matrix of normal means with unknown covariance matrix. J. Multi. Analysis 36:44–55) have shown that their shrinkage estimators perform better than the least squares estimator. Recently, Breiman and Friedman (Breiman, L., Friedman, J. H. (1997). Predicting multivariate responses in multiple regression. J. Roy. Statist. Soc. Ser. B 59:3–54) proposed another class of shrinkage estimators, called C&W-GCV estimators. Through extensive simulations, they have showed that their C&W-GCV estimator performs better than the FICYREG estimator of van der Merwe and Zidek (van der Merwe, A., Zidek, J. V. (1980). Multivariate regression analysis and canonical variates. Canadian Journal of Statistics 8:27–39), the reduced rank regression method of Anderson (Anderson, T. W. (1951). Estimating linear restrictions on regression coefficients for multivariate normal distribution. Ann. Math. Statist., 22:327–351 (Correction in Ann. Statist. (1980), 8, 1400). Estimating linear restrictions on regression coefficients for multivariate normal distribution. Ann. Math. Statist. 22:327–351. (Correction in Ann. Statist. (1980), 8, 1400)), the component-wise ridge regression and the partial least squares. They, however, did not include in their comparisons, the minimax estimators of Bilodeau and Kariya (Bilodeau, M., Kariya, T. (1989). Minimax estimators in the normal MANOVA model. Journal of Multivariate Analysis 28:260–270) and Konno (Konno, Y. (1990). On estimation of a matrix of mean. Unpublished manuscript; Konno, Y. (1991). On estimation of a matrix of normal means with unknown covariance matrix. J. Multi. Analysis 36:44–55). In this article, we compare C&W-GCV estimator with two invariant minimax estimators and show that C&W-GCV does not perform as well as the two minimax estimators unless the number of response variables is fairly small compared to the number of independent variables and the sample size is small.

Keywords:

• Multivariate response
• Linear regression model
• Shrinkage estimators
• Simulations

Acknowledgments

The first author would like to thank Professor Rolf Sundberg for stimulating discussion during his visit to Stockholm in the summer of 1996. Thanks are also due to Professor Tatsuya Kubokawa for valuable discussion and to a referee for helpful suggestions. The research of the first author was supported partially by Swedish Research Council and Natural Sciences of Engineering Research Council of Canada. The research of the second author was partially supported by the LEQSF grant number RD-A-31.