Optimal Model Averaging of Mixed-Data Kernel-Weighted Spline Regressions

Abstract

Model averaging has a rich history dating from its use for combining forecasts from time-series models (Bates and Granger) and presents a compelling alternative to model selection methods. We propose a frequentist model averaging procedure defined over categorical regression splines (Ma, Racine, and Yang) that allows for mixed-data predictors, as well as nonnested and heteroscedastic candidate models. We demonstrate the asymptotic optimality of the proposed model averaging estimator, and develop a post-averaging inference theory for it. Theoretical underpinnings are provided, finite-sample performance is evaluated, and an empirical illustration reveals that the method is capable of outperforming a range of popular model selection criteria in applied settings. An R package is available for practitioners (Racine).

keyword:

• Categorical covariates
• Cross-validation
• Frequentist estimation
• Nonparametric estimation

Supplementary Materials

Supplementary materials include proofs of the main theorems in the paper along with tabular summaries of the Monte Carlo simulations.

Disclosure Statement

The authors report there are no competing interests to declare.

Acknowledgments

The authors would like to thank the Editor, Associate Editor, and two anonymous reviewers for their constructive suggestions and comments that have substantially improved earlier versions of this article. All authors equally contribute to the article. All remaining errors are solely ours. Li Zheng is the corresponding author of this article.

Additional information

Funding

Dalei Yu was supported by National Natural Science Foundation of China (NNSFC, grant nos. 12071414 and 11661079), Li Zheng was supported by NNSFC, grant no. 72203076, and Jeffrey Racine was supported by the Social Sciences and Humanities Research Council of Canada.