Coding the Output of a Feedforward Neural Net

Abstract

In any feedforward neural net, there are many choices for coding, or labeling, the input data. Error-correcting codes have been proposed to encode the neural net output. This representation adds extra distance between the labels of the different classes, thus it helps to combat classification errors encountered in feedforward nets. This claim is verified theoretically and some useful bounds are derived to characterize the error-correcting code for such applications. The overhead of coding is to require more output nodes, i.e. a more complex network. It is known that, in general a more complex network has more capacity. Thus, we investigate the capacity and separation ability of the coded network and compare the effect of coding the output with that of using a specific two-layered net. This comparison is carried out from a deterministic and then from a probabilistic view point. The issues of finding a neural net decoder is also addressed and analyzed. This leads to a new look at the multi-layer neural net which helps in finding an upper bound on the complexity of the multi-layer neural net.

Keywords:

• Keywords: Output-coded Neural Net
• Neural Net Capacity
• Error Correction In Neural Nets
• Error-tolerant Neural Nets