TAP equation for non-negative Boltzmann machine

Abstract

Mean-field methods for spin systems are frequently used in not only statistical physics but also information sciences. We focus on the Plefka expansion method for spin systems with two-body interactions. The Plefka expansion is a useful perturbative expansion of the Gibbs free energy, and it can systematically provide the naive mean-field approximation, the Thouless–Anderson–Palmer (TAP) equation and higher-order approximations. In the first part of this paper, using the linear response relation, we derive a recurrence formula for perturbative coefficients in the Plefka expansion. Our recurrence formula enables us to systematically derive general order coefficients. In the latter part of the paper, we apply our recurrence formula to the non-negative Boltzmann machine in which all spin variables are constrained to have non-negative real values, and we obtain the TAP equation for this model. We verify the performance of our TAP equation by conducting some numerical experiments.

Keywords:

• Markov random fields
• mean-field methods
• Plefka expansion
• TAP equation
• linear response relation
• non-negative Boltzmann machines

Acknowledgements

This work was partly supported by Grants-In-Aid (No. 21700247 and No. 22300078) for Scientific Research from the Ministry of Education, Culture, Sports, Science and Technology, Japan.

Notes

1. Downs’ expansion is obtained by

The extremum condition of this expansion with respect to m at α = 1 gives