Self-organization of Multiple Winner-take-all Neural Networks

Abstract

In this paper, analysis of the information content of discretely firing neurons in unsupervised neural networks is presented, where information is measured according to the network's ability to reconstruct its input from its output with minimum mean square Euclidean error. It is shown how this type of network can self-organize into multiple winner-take-all subnetworks, each of which tackles only a low-dimensional subspace of the input vector. This is a rudimentary example of a neural network that effectively subdivides a task into manageable subtasks.

Keywords:

• Keywords: Self-organization
• Firing Neurons
• Winner-take-all
• Factorial Network
• Ocular Dominance