Generalisation and discrimination emerge from a self-organising componential network: a speech example

Abstract

It is demonstrated that a componential code emerges when a self-organising neural network is exposed to continuous speech. The code's components correspond to substructures that occur relatively independently of one another: words and phones. A capability for generalisation and discrimination develops without having been optimised explicitly. The componential structure is revealed by optimising a necessarily complicated nonlinear moment of the data's distribution, equal to the mean-squared output response of a multi-layered network of simple threshold neurons. Earlier analytical work had predicted that componential codes, generalisation and discrimination should emerge from the self-organisation of threshold neurons of this form, assuming certain properties of the pattern-space distribution of the data.