Improving learning in neural networks through weight initializations

Abstract

Feedforward neural networks are known to exhibit universal approximation property, i.e., these networks can effectively approximate any function arbitrary well, provided there is one hidden layer with sufficient number of hidden nodes, and some non-linearity at the hidden layer nodes. The property of universal approximation in these networks contributed to wide range of application, remarkably in the area of predictive modelling. However, these networks have a major drawback of slow convergence. In this work, we proposed a new weight initialization algorithm to improve the convergence speed in feedforward networks using statistical analysis of system, based on the set of predefined assumptions. The proposed algorithm is compared to other established weight initialization algorithm for predictive models, and is expected to perform better than other algorithms.

Subject Classification:

• 92B20 (Neural networks, artificial life and related topics)

Keywords:

• Initial weight choice problem
• Feedforward networks
• Function approximation