Smoothing Spline Semiparametric Density Models

Abstract

Density estimation plays a fundamental role in many areas of statistics and machine learning. Parametric, nonparametric, and semiparametric density estimation methods have been proposed in the literature. Semiparametric density models are flexible in incorporating domain knowledge and uncertainty regarding the shape of the density function. Existing literature on semiparametric density models is scattered and lacks a systematic framework. In this article, we consider a unified framework based on reproducing kernel Hilbert space for modeling, estimation, computation, and theory. We propose general semiparametric density models for both a single sample and multiple samples which include many existing semiparametric density models as special cases. We develop penalized likelihood based estimation methods and computational methods under different situations. We establish joint consistency and derive convergence rates of the proposed estimators for both finite dimensional Euclidean parameters and an infinite-dimensional functional parameter. We validate our estimation methods empirically through simulations and an application. Supplementary materials for this article are available online.

Keywords:

• Backfitting
• Gauss–Newton algorithm
• Joint asymptotics
• Multi-samples
• Nonlinear models
• Reproducing kernel Hilbert space

Acknowledgments

The authors would also like to thank the associate editor and two referees for constructive comments that substantially improved an earlier draft.

Additional information

Funding

This research is supported by National Science Foundation grants DMS-1507078 for Anna Liu, and DMS-1507620 for Yuedong Wang. The authors gratefully acknowledge support from the Center for Scientific Computing from the CNSI, MRL: an NSF MRSEC (DMR-1720256).