SCAD-penalised generalised additive models with non-polynomial dimensionality

Abstract

In this article, we study the (group) smoothly clipped absolute deviation (SCAD) estimator in the estimation of generalised additive models. The SCAD penalty, proposed by Fan and Li [(2001) ‘Variable Selection via Nonconcave Penalised Likelihood and Its Oracle Properties’, Journal of the American Statistical Association 96(456), 1348–1360], has many desirable properties including continuity, sparsity and unbiasedness. For high-dimensional parametric models, it has only recently been shown that the SCAD estimator can deal with problems with dimensions much larger than the sample size. Here, we show that the SCAD estimator can be successfully applied to generalised additive models with non-polynomial dimensionality and our study represents the first such result for the SCAD estimator in nonparametric problems, as far as we know. In particular, under suitable assumptions, we theoretically show that the dimension of the problem can be close to exp<texlscub>n ^d/(2d+1)</texlscub>, where n is the sample size and d is the smoothness parameter of the component functions. Some Monte Carlo studies and a real data application are also presented.

Keywords:

• nonparametric oracle property
• polynomial splines
• quasi-likelihood
• SCAD penalty

Acknowledgements

We thank the AE and two anonymous referees for their helpful comments that improved the manuscript. Gaorong Li's research was supported by the NNSF (11101014,11002005) of China, the Specialised Research Fund for the Doctoral Program of Higher Education of China (20101103120016), PHR(IHLB, PHR20110822), the Training Programme Foundation for the Beijing Municipal Excellent Talents (2010D005015000002) and the Fundamental Research Foundation of Beijing University of Technology (X4006013201101). Heng Lian's research was supported by Singapore Ministry of Education Tier 1 Grant.