Two-Way Truncated Linear Regression Models with Extremely Thresholding Penalization

Abstract

This article introduces a new type of linear regression model with regularization. Each predictor is conditionally truncated through the presence of unknown thresholds. The new model, called the two-way truncated linear regression model (TWT-LR), is not only viewed as a nonlinear generalization of a linear model but is also a much more flexible model with greatly enhanced interpretability and applicability. The TWT-LR model performs classifications through thresholds similar to the tree-based methods and conducts inferences that are the same as the classical linear model on different segments. In addition, the innovative penalization, called the extremely thresholding penalty (ETP), is applied to thresholds. The ETP is independent of the values of regression coefficients and does not require any normalizations of regressors. The TWT-LR-ETP model detects thresholds at a wide range, including the two extreme ends where data are sparse. Under suitable conditions, both the estimators for coefficients and thresholds are consistent, with the convergence rate for threshold estimators being faster than $\sqrt{n}$. Furthermore, the estimators for coefficients are asymptotically normal for fixed dimension p. It is demonstrated in simulations and real data analyses that the TWT-LR-ETP model illustrates various threshold features and provides better estimation and prediction results than existing models. Supplementary materials for this article are available online.

Keywords:

• Asymptotics
• Consistency
• Extremely thresholding
• High dimension
• Interpretability
• Predictability
• Thresholds

Supplementary Materials

The supplementary materials contain codes, 4 datasets used in the article, a readme file and an online supplement containing theoretical justifications, simulation results and real data results.

Acknowledgments

The authors thank the editor, the associate editor, and two anonymous referees for their tremendous efforts and insightful comments that substantially improve the article’s quality and presentation.

Additional information

Funding

The partial support from NSF-DMS-2012298 is also acknowledged.