Adaptation of Learning Rule Parameters Using a Meta Neural Network

Abstract

This paper proposes an application-independent method of automating learning rule parameter selection using a form of supervisor neural network (NN), known as a meta neural network (MNN), to alter the value of a learning rule parameter during training. The MNN is trained using data generated by observing the training of a NN and recording the effects of the selection of various parameter values. The MNN is then combined with a normal learning rule to augment its performance. Experiments are undertaken to see how this method performs by using it to adapt a global parameter of the resilient backpropagation and quickpropagation learning rules.

Keywords:

• Supervisor Network
• Resilient Backpropagation
• Quickpropagation