Guided Regression Modeling for Prediction and Exploration of Structure With Many Explanatory Variables

Abstract

A modeling procedure for multiple linear regression is proposed. This procedure begins with preliminary interior and global analyses. The global analysis is based on a form of canonical analysis of the sample correlation matrix of all variables, and, depending on the regression objective, the procedure uses information from that analysis as a guide in the selection of methods to achieve the objective. The two objectives discussed are prediction and exploration of structure. The dependence of the choice of methods on the regression objective is illustrated on a “benchmark” data set, and the results obtained by our approach are compared with published results obtained by other methods. The procedure suggested is particularly useful for data sets with large numbers of explanatory variables that render more conventional methods more expensive, less flexible, or less informative concerning relationships among variables.

KEY WORDS:

• Multiple regression
• Collinearity
• Prediction
• Variable selection
• Cross validation
• Biased regression