Fully Data-Driven Normalized and Exponentiated Kernel Density Estimator with Hyvärinen Score

Abstract

Recently, Jewson and Rossell proposed a new approach for kernel density estimation using an exponentiated form of kernel density estimators. The density estimator contained two hyperparameters that flexibly controls the smoothness of the resulting density. We tune them in a data-driven manner by minimizing an objective function based on the Hyvärinen score to avoid the optimization involving the intractable normalizing constant caused by the exponentiation. We show the asymptotic properties of the proposed estimator and emphasize the importance of including the two hyperparameters for flexible density estimation. Our simulation studies and application to income data show that the proposed density estimator is promising when the underlying density is multi-modal or when observations contain outliers.

KEYWORDS:

• Bandwidth selection
• Density estimation
• Fisher divergence
• Kernel smoothing
• Unnormalized model

Supplementary Materials

The Supplementary Materials contain proof of theoretical results, additional results and numerical analysis. In particular, the asymptotic expansion of the MISE of the extended KDE (Theorem S1) is a result that may be of interest in the context of non-parametric estimation beyond the context of generalized Bayesian and non-normalized models.

Disclosure Statement

The authors report there are no competing interests to declare.

Additional information

Funding

This study was supported by JSPS KAKENHI (grant no. 21H00699 and 23H00805).