DDAC-SpAM: A Distributed Algorithm for Fitting High-dimensional Sparse Additive Models with Feature Division and Decorrelation

Abstract

Abstract–

Distributed statistical learning has become a popular technique for large-scale data analysis. Most existing work in this area focuses on dividing the observations, but we propose a new algorithm, DDAC-SpAM, which divides the features under a high-dimensional sparse additive model. Our approach involves three steps: divide, decorrelate, and conquer. The decorrelation operation enables each local estimator to recover the sparsity pattern for each additive component without imposing strict constraints on the correlation structure among variables. The effectiveness and efficiency of the proposed algorithm are demonstrated through theoretical analysis and empirical results on both synthetic and real data. The theoretical results include both the consistent sparsity pattern recovery as well as statistical inference for each additive functional component. Our approach provides a practical solution for fitting sparse additive models, with promising applications in a wide range of domains. Supplementary materials for this article are available online.

KEYWORDS:

• Additive model
• Consistency
• Decorrelate and conquer
• Divide
• Feature space partition
• Variable selection

Supplementary Materials

The supplementary material consists of Lemma S.1–S.6 and the proofs of all lemmas, theorems, and corollaries.

Acknowledgments

We thank the editor, the AE, and anonymous reviewers for their insightful comments which have greatly improved the scope and quality of the article.

Disclosure Statement

The authors report there are no competing interests to declare.

Additional information

Funding

Zhou was supported by the State Key Program of National Natural Science Foundation of China (71931004) and National Natural Science Foundation of China (92046005) and the National Key R&D Program of China (2021YFA1000100, 2021YFA1000101). Feng was supported by NIH grant 1R21AG074205-01, NYU University Research Challenge Fund, a grant from NYU School of Global Public Health, and through the NYU IT High Performance Computing resources, services, and staff expertise.