A novel weighted likelihood estimation with empirical Bayes flavor

ABSTRACT

We propose a novel approach to estimation, where a set of estimators of a parameter is combined into a weighted average to produce the final estimator. The weights are chosen to be proportional to the likelihood evaluated at the estimators. We investigate the method for a set of estimators obtained by using the maximum likelihood principle applied to each individual observation. The method can be viewed as a Bayesian approach with a data-driven prior distribution. We provide several examples illustrating the new method and argue for its consistency, asymptotic normality, and efficiency. We also conduct simulation studies to assess the performance of the estimators. This straightforward methodology produces consistent estimators comparable with those obtained by the maximum likelihood method. The method also approximates the distribution of the estimator through the “posterior” distribution.

KEYWORDS:

• Consistency
• Data-dependent prior
• Empirical Bayes
• Exponentiated distribution
• Maximum likelihood estimator
• Super-efficiency
• Unbounded likelihood

MATHEMATICS SUBJECT CLASSIFICATION:

• 62C12, 62F10, 62F12, 62F15

Acknowledgments

We thank the two anonymous referees for their comments, which helped us streamline the exposition and motivated us to derive asymptotic normality and efficiency of the presented methodology. The research of Podgórski and Kozubowski has been partially supported by the Riksbankens Jubileumsfond Grant Dnr: P13-1024:1.

Additional information

Funding

Riksbankens Jubileumsfond(10.13039/501100004472, P13-1024:1)