Density estimation via the random forest method

ABSTRACT

The problem of density estimation arises naturally in many contexts. In this paper, we consider the approach using a piecewise constant function to approximate the underlying density. We present a new density estimation method via the random forest method based on the Bayesian Sequential Partition (BSP) (Lu, Jiang, and Wong 2013). Extensive simulations are carried out with comparison to the kernel density estimation method, BSP method, and four local kernel density estimation methods. The experiment results show that the new method is capable of providing accurate and reliable density estimation, even at the boundary, especially for i.i.d. data. In addition, the likelihood of the out-of-bag density estimation, which is a byproduct of the training process, is an effective hyperparameter selection criterion.

KEYWORDS:

• Density estimation
• error estimation
• random forest.

MATHEMATICS SUBJECT CLASSIFICATION:

•  

Acknowledgments

We are grateful to the reviewer for his/her suggestions that lead to substantial improvement of the paper. We are grateful to Arup Bose and Santanu Dutta for sharing their work and Dr. Wei Wei for helpful discussions.