A Note on High-Dimensional Linear Regression With Interactions

ABSTRACT

The problem of interaction selection in high-dimensional data analysis has recently received much attention. This note aims to address and clarify several fundamental issues in interaction selection for linear regression models, especially when the input dimension p is much larger than the sample size n. We first discuss how to give a formal definition of “importance” for main and interaction effects. Then we focus on two-stage methods, which are computationally attractive for high-dimensional data analysis but thus far have been regarded as heuristic. We revisit the counterexample of Turlach and provide new insight to justify two-stage methods from the theoretical perspective. In the end, we suggest new strategies for interaction selection under the marginality principle and provide some simulation results.

KEYWORDS:

• Heredity condition
• Hierarchical structure
• Interaction effects
• Linear model
• Marginality principle

Additional information

Funding

The authors gratefully acknowledge the funding support of NSF DMS-1309507, NSF DMS-1418172, and NSFC-11571009.