Resampling Methods for Model Fitting and Model Selection

Abstract

Resampling procedures for fitting models and model selection are considered in this article. Nonparametric goodness-of-fit statistics are generally based on the empirical distribution function. The distribution-free property of these statistics does not hold in the multivariate case or when some of the parameters are estimated. Bootstrap methods to estimate the underlying distributions are discussed in such cases. The results hold not only in the case of one-dimensional parameter space, but also for the vector parameters. Bootstrap methods for inference, when the data is from an unknown distribution that may or may not belong to a specified family of distributions, are also considered. Most of the information criteria-based model selection procedures such as the Akaike information criterion, Bayesian information criterion, and minimum description length use estimation of bias. The bias, which is inevitable in model selection problems, arises mainly from estimating the distance between the “true” model and an estimated model. A jackknife type procedure for model selection is discussed, which instead of bias estimation is based on bias reduction.

Key Words:

• Information criterion
• Jackknife
• Kullback–Leibler divergence
• Maximum likelihood estimation
• Model selection
• Nonparametric bootstrap
• Parametric bootstrap

ACKNOWLEDGMENT

The work was supported in part by National Science Foundation grant AST-1047586.