On divergences of finite measures and their applicability in statistics and information theory

Abstract

Modifications of the classical φ-divergences D _φ(μ, ν)=∈t q φ (p/q) dλ of finite measures μ, ν on a σ-finite measure space (𝒳, 𝒜, λ) with Radon–Nikodym densities p=dμ/dλ, q=dν/dλ are introduced by the formula 𝔇_φ(μ, ν)=∈t q φ˜ (p/q) dλ using the nonnegative convex functions . Basic properties of the modified φ-divergences are investigated, such as the range of values, symmetry and a decomposition into local and global components. A general φ-divergence formula for right-censored observations illustrates the statistical applicability. The Pinsker inequality for finite measures and the generalized Ornstein distance of stationary random processes are among the illustrations of applicability in the information theory.

Keywords:

• divergences of finite measures
• local and global divergences of finite measures
• divergences of σ-finite measures
• statistical censoring
• Pinsker's inequality for finite measures
• differential power entropies

AMS Subject Classification :

• 62B10
• 94A17
• 62N01
• 62C10

Acknowledgements

We are very grateful to two unknown referees for their valuable suggestions and comments. We also thank for the partial support by the MŠMT grant 1M0572 and the GAČR grant 102/07/1131.