A closed-measure approach to stochastic approximation

Abstract

This paper introduces a new method to tackle the issue of the almost sure convergence of stochastic approximation algorithms defined from a differential inclusion. Under the assumption of slowly decaying step-sizes, we establish that the set of essential accumulation points of the iterates belongs to the Birkhoff centre associated with the differential inclusion. Unlike previous works, our results do not rely on the notion of asymptotic pseudotrajectories, predominant technique to address the convergence problem. They follow as a consequence of Young's superposition principle for closed measures. This perspective bridges the gap between Young's principle and the notion of invariant measure of set-valued dynamical systems introduced by Faure and Roth. Also, the proposed method allows for obtaining sufficient conditions under which the velocities locally compensate around any essential accumulation point.

KEYWORDS:

• Stochastic approximation
• closed measures
• weak convergence
• differential inclusions

Acknowledgments

The authors are grateful for numerous enlightening discussions with Jérôme Bolte and Edouard Pauwels. This line of research grew out of inspiring conversations with Sylvain Sorin.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The second author wishes to acknowledge the generous support of ANR-3IA Artificial and Natural Intelligence Toulouse Institute [ANR-19-PI3A-0004], and of the Air Force Office of Scientific Research, Air Force Material Command, usaf, under grant number FA9550-19-1-7026, as well as partial funding from UiT Aurora Center mascot; Ministry of Education, Youth and Sports of the Czech Republic [CZ.02.01.01/00/22_008/0004590].