A search for the optimal thresholding sequence in an associative memory

Abstract

In learning matrix associative memory networks, the choice of threshold value is one of the most significant factors for determining the recall performance. Choice of threshold is especially important for multi-step recall, as each network state is dependent on the prior states. Recently, Gibson and Robinson used a statistical approximation to formalize the dynamics of partially connected recurrent networks. Using this formalism, we evaluate all of the possible thresholding sequences and find the sequence that results in the highest storage capacity. The resulting optimal strategy can be closely approximated by a threshold that is proportional to the activation of the network plus an offset. The performance of a simulated associative memory is shown to match well with the predictions of the theory. This strategy corresponds to one of the simplest putative roles of interneurons which provide a linear output proportional to the total unweighted input from the principal cells.