Statistical analysis of the expectation-maximization algorithm with loopy belief propagation in Bayesian image modeling

Abstract

New statistical approaches to hyperparameter estimation by means of the expectation-maximization (EM) algorithm and loopy belief propagation (LBP) are given for Bayesian image modeling from the standpoint of statistical-mechanical informatics. In the present paper, we give a new scheme for computing the average of the trajectory in the EM algorithm with LBP with respect to all the possible degraded images generated by the assumed degradation process from a given original image. Moreover, we also give a new scheme for computing the average of the trajectory in the EM algorithm with LBP with respect to all the possible original images generated by the assumed prior probability distribution. Bayesian image models for binary image restorations are constructed as Ising models with random external fields and spatially uniform nearest-neighbor interactions on the square lattice. Our schemes for computing the above random averages for some statistical quantities in LBP are constructed by means of the Bethe approximation for random spin systems. They are reduced to the solution of simultaneous integral equations for distributions of messages in LBP. We show the results of some numerical experiments in LBP as well as the statistical analysis of the trajectory in the EM algorithm for binary image restoration by LBP.

Keywords:

• Bayesian statistics
• EM algorithm
• maximum likelihood estimation
• probabilistic image processing
• Markov random fields
• Bethe approximation
• loopy belief propagation

Acknowledgments

This work was partly supported by the Grants-In-Aid (No.22300078, No.21700247) for Scientific Research from the Ministry of Education, Culture, Sports, Science and Technology of Japan.