Generalised correlated cross-validation

Abstract

Since its introduction by [Stone, M. (1974), ‘Cross-validatory Choice and the Assessment of Statistical Predictions (with discussion)’, Journal of the Royal Statistical Society, B36, 111–133] and [Geisser, S. (1975), ‘The Predictive Sample Reuse Method with Applications’, Journal of the American Statistical Association, 70, 320–328], cross-validation has been studied and improved by several authors including [Burman, P., Chow, E., and Nolan, D. (1994), ‘A Cross-validatory Method for Dependent Data’, Biometrika, 81(2), 351–358], [Hart, J. and Yi, S. (1998), ‘One-sided Cross-validation’, Journal of the American Statistical Association, 93(442), 620–630], [Racine, J. (2000), ‘Consistent Cross-validatory Model-selection for Dependent Data: hv-block Cross-validation’, Journal of Econometrics, 99, 39–61], [Hart, J. and Lee, C. (2005), ‘Robustness of One-sided Cross-validation to Autocorrelation’, Journal of Multivariate Analysis, 92(1), 77–96], and [Carmack, P., Spence, J., Schucany, W., Gunst, R., Lin, Q., and Haley, R. (2009), ‘Far Casting Cross Validation’, Journal of Computational and Graphical Statistics, 18(4), 879–893]. Perhaps the most widely used and best known is generalised cross-validation (GCV) [Craven, P. and Wahba, G. (1979), ‘Smoothing Noisy Data with Spline Functions’, Numerical Mathematics, 31, 377–403], which establishes a single-pass method that penalises the fit by the trace of the smoother matrix assuming independent errors. We propose an extension to GCV in the context of correlated errors, which is motivated by a natural definition for residual degrees of freedom. The efficacy of the new method is investigated with a simulation experiment on a kernel smoother with bandwidth selection in local linear regression. Next, the winning methodology is illustrated by application to spatial modelling of fMRI data using a nonparametric semivariogram. We conclude with remarks about the heteroscedastic case and a potential maximum likelihood framework for Gaussian random processes.

Keywords:

• effective degrees of freedom
• fMRI
• model selection
• nonparametric
• spatial semivariogram
• supervised learning
• tuning parameter