Mixed integer nonlinear goal programming approach to variable selection in linear regression

Abstract

Several methods have been proposed in regression analysis for variable selection. Such methods enable researchers to distinguish between significant and insignificant variables, thus they provide a parsimonious regression model based solely on significant variables. Classical methods for variable selection include forward selection, backward elimination, and stepwise regression. The use of these methods is restricted by some strict assumptions about the given data. In the current study a new Mixed Integer Nonlinear Goal Programing model, which dispenses with these rigid assumptions, is introduced. The new model is less sensitive to outliers and allows incorporating any necessary restrictions on the model’s parameters. The performance of the new model is compared with the Classical methods using large scale simulation studies. The evaluation of the proposed method and the Classical methods will be done based on the proportion of correct selection of the significant independent variables.

Keywords:

• Mixed integer nonlinear goal programing (MINGP)
• Forward selection method
• Backward elimination method
• Stepwise regression

Acknowledgments

This Project was funded by the Deanship of Scientific Research (DSR) at King Abdulaziz University, Jeddah, under grant no. (G-98-130-38). The authors, therefore, acknowledge with thanks DSR for technical and financial support. The authors, also, would like to express their deep thanks and appreciation to Dr. Sherif S. Ali for his great efforts in programing for the simulation studies.