Electronic implementation of an analogue attractor neural network with stochastic learning

Abstract

We describe and discuss an electronic implementation of an attractor neural network with plastic synapses. The network undergoes double dynamics, for the neurons as well as the synapses. Both dynamical processes are unsupervised. The synaptic dynamics is autonomous, in that it is driven exclusively and perpetually by neural activities. The latter follow the network activity via the developing synapses and the influence of external stimuli. Such a network self-organizes and is a device which converts the gross statistical characteristics of the stimulus input stream into a set of attractors (reverberations). To maintain for long time the acquired memory, the analog synaptic efficacies are discretized by a stochastic refresh mechanism. The discretized synaptic memory has indefinitely long lifetime in the absence of activity in the network. It is modified only by the arrival of new stimuli. The stochastic refresh mechanism produces transitions at low probability which ensures that transient stimuli do not create significant modifications and that the system has large palimpsestic memory. A change in the attractor structure represents a major, macroscopic change in the statistics of the input stream, which may deform attractors, may create new ones and may eliminate others.

The electronic implementation is completely analogue, stochastic and asynchronous. The circuitry of the first prototype is discussed in detail as well as the tests performed on it. In carrying out the implementation we have been guided by biological considerations and by electronic constraints. Both are discussed and new insights and lessons for the learning process are proposed.