A Subsampling Method for Regression Problems Based on Minimum Energy Criterion

Abstract

The extraordinary amounts of data generated nowadays pose heavy demands on computational resources and time, which hinders the implementation of various statistical methods. An efficient and popular strategy of downsizing data volumes and thus alleviating these challenges is subsampling. However, the existing methods either rely on specific assumptions for the underlying models or acquire partial information from the available data. For regression problems, we propose a novel approach, termed adaptive subsampling with the minimum energy criterion (ASMEC). The proposed method requires no explicit model assumptions and “smartly” incorporates information on covariates and responses. ASMEC subsamples possess two desirable properties: space-fillingness and spatial adaptiveness. We investigate the limiting distribution of ASMEC subsamples and their theoretical properties under the smoothing spline regression model. The effectiveness and robustness of the ASMEC approach are also supported by a variety of synthetic examples and two real-life examples.

KEYWORDS:

• Basis selection
• Massive data
• Smoothing spline
• Space-filling
• Spatial adaptiveness

Supplementary Materials

Supplementary Material.pdf: The file supplements more details of the smoothing spline regression model, proofs of Theorems 1–3, algorithm of ASMEC subsampling for higher-dimensional data, and an illustration of functions used in the simulations of the main manuscript.

ASMEC code.zip: The file includes code to perform the proposed methods and all datasets used as examples in the article.

Acknowledgments

The authors sincerely thank the editor, an associated editor and two anonymous referees for their valuable comments and suggestions.

Disclosure Statement

The authors report there are no competing interests to declare.

Funding

Additional information

Funding

The research of Wenlin Dai and Yan Song was supported by the National Natural Science Foundation of China (grant no. NSFC 11901573) and the Beijing Natural Science Foundation (Z200001). The research of Dianpeng Wang was supported by the National Natural Science Foundation of China (grant no. NSFC 11801034 and grant no. NSFC 12171033).