Additive regression splines with total variation and non negative garrote penalties

Abstract

This study examines a penalized additive regression spline estimator with total variation and non negative garrote-type penalties. The proposed estimator is obtained based on a two-stage procedure. In the first stage, an initial estimator is obtained via total variation penalization. The total variation penalty enables data-adaptive knot selection and regularizes the overall smoothness of the estimator. The second stage imposes the non negative garrote penalty on the estimated functional components to attain variable selectivity. Regarding the theoretical aspect, a non asymptotic oracle inequality for the total variation penalized estimator is established under some regularity conditions. Based on the oracle inequality, we prove that the estimator attains the optimal rate of convergence up to a logarithmic factor, which in turn leads to the selection and estimation consistency of the second-stage garrote estimator. Numerical studies are presented to illustrate the usefulness of a combination of these two penalties. The results show that the proposed method outperforms existing methods.

Keywords:

• Coordinate descent algorithm
• model selection
• non parametric smoothing
• oracle inequality
• penalized least squares

Additional information

Funding

The research of Ja-Yong Koo was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (NRF-2018R1D1A1B07049972). The research of Jae-Hwan Jhong was supported by a Korea University Grant and the NRF (NRF-2020R1G1A1A01100869).