Computational efficiency in all possible regressions

Abstract

In this paper, we develop an efficient procedure in identifying subset for which the residual sum of squares is minimum among subsets with the same number of input variables. The suggested procedure is a combination of works, by Hocking and Leslie (1967), and Newton and Spurrell (1967a), which both identify the best subsets by computing only a small fraction of the all possible regressions. A ten-variable example demonstrates that the proposed algorithm significantly reduces the number of subsets required to obtain the best subsets.

Keywords:

• linear regression
• all possible regression
• residual sum of squares
• extra regression sum of squares