Distribution-Free Prediction Sets

Abstract

This article introduces a new approach to prediction by bringing together two different nonparametric ideas: distribution-free inference and nonparametric smoothing. Specifically, we consider the problem of constructing nonparametric tolerance/prediction sets. We start from the general conformal prediction approach, and we use a kernel density estimator as a measure of agreement between a sample point and the underlying distribution. The resulting prediction set is shown to be closely related to plug-in density level sets with carefully chosen cutoff values. Under standard smoothness conditions, we get an asymptotic efficiency result that is near optimal for a wide range of function classes. But the coverage is guaranteed whether or not the smoothness conditions hold and regardless of the sample size. The performance of our method is investigated through simulation studies and illustrated in a real data example.

Keywords:

• Conformal prediction
• Consistency
• Density level sets
• Finite sample
• Kernel density

Acknowledgments

Jing Lei’s work is supported by the National Science Foundation grant BCS-0941518 and Larry Wasserman’s work by the National Science Foundation grant DMS-0806009 and Air Force grant FA95500910373.