The Mean Relative Entropy: An Invariant Measure of Estimation Error

Abstract

A fundamental issue in statistics is parameter estimation, where the first step is to select estimators under some measure of estimation error. The commonly used measure is the mean squared error, which is simple, intuitive and highly interpretable, but it has some drawbacks, often creating confusions in evaluating estimators. To solve these problems, we propose two invariance properties and the sufficiency principle as the prerequisite for any reasonable measure. Then, the mean relative entropy is established as an invariant measure of estimation error.

Keywords:

• Invariance property
• Mean squared error
• Parameter estimation
• Sufficiency principle
• Transformation

Acknowledgments

This research is supported by the Natural Science Foundation of China (NSFC, Grant No. 11561073, 11461079). The author thanks two referees, an associate editor, and the editor for their helpful comments, corrections and suggestions.