Removing potential flat spots on error surface of multilayer perceptron (MLP) neural networks

References

• Bengio , Y. , Le Roux , N. , Vincent , P. , Delalleau , O. and Marcotte , P. 2005 . “ Convex neural networks ” . In Neural Information Processing Systems (NIPS 2005) Vancouver, , British Columbia, Canada
   Google Scholar
• Bi , W. , Wang , X. G. , Tang , Z. and Tamura , H. 2005 . Avoiding the local minima problem in backpropagation algorithm with modified error function . IEICE Trans. Fundam. , 88 ( 12 ) : 3645 – 3653 .
   Google Scholar
• Blake , C. L. and Merz , C. J. 1998 . “ UCI Repository of machine learning data bases ” . Irvine, CA : University of California, Department of information and computer science . Available at http://www.ics.uci.edu/~mlearn/MLRepository.html
   Google Scholar
• Fisher , R. A. 1936 . The use of multiple measurements in taxonomic problems . Ann. Eugen. , 7 : 179 – 188 .
   Google Scholar
• Goerick , C. and Seelen , W. V. On unlearnable problems or a model for premature saturation in backpropagation learning . Proceedings of the European Symposium on Artificial Neural Networks . April 24–26 , Brugge, Belgium. pp. 13 – 18 .
   Google Scholar
• Gorman , R. P. and Sejnowski , T. J. 1988 . Analysis of hidden units in a layered network trained to classify sonar targets . Neural Netw. , 1 : 75 – 89 .
   Web of Science ®Google Scholar
• Gori , M. and Tesi , A. 1992 . On the problem of local minima in backpropagation . IEEE Trans. Pattern Anal. Mach. Intell. , 14 ( 1 ) : 76 – 86 .
   Web of Science ®Google Scholar
• Graupe , D. 2007 . “ Principles of Artificial Neural Networks ” . In Advanced Series in Circuits and Systems , Vol. 6 , Singapore : World Scientific Publishing .
   Google Scholar
• Haykin , S. 1994 . Neural Networks, a Comprehensive Foundation , USA : MacMillan Publishing .
   Google Scholar
• Moallem , P. and Ayoughi , S. A. Increasing convergence rate in training multi layer perceptron by modifying dynamic self adaptation algorithm . 16th Iranian Conference on Electrical Engineering (ICEE 2008) . Tehran, Iran.
   Google Scholar
• Moallem , P. and Kyoumarsi , A. PTGVLR: Fast MLP learning using parallel tangent gradient with variable learning rates . Proceedings of the IEEE International Conference on Control, Automation and Systems . October 17–20 , Seoul, Korea. pp. 2162 – 2165 .
   Google Scholar
• Nawi , N. M. , Ransing , R. S. and Ransing , M. R. 2007 . An improved conjugate gradient based learning algorithm for back propagation neural networks . Int. J. Comput. Intell. , 4 ( 1 ) : 46 – 55 .
   Google Scholar
• Parekh , R. , Balakrishnan , K. and Honavar , V. An empirical comparison of flat-spot elimination techniques in back-propagation networks . 3rd Workshop on Neural Networks . Alabama, USA.
   Google Scholar
• Rumelhart , D. E. , Hinton , G. E. and Williams , R. J. 1986 . “ Learning internal representations by back-propagation errors ” . In Parallel Distributed Processing: Explorations in the Microstructure of Cognition , Edited by: Rumelhart , D. E. , McClelland , J. L. and the PDP Research Group . Vol. 1 , 318 – 362 . Cambridge, MA : MIT Press .
   Google Scholar
• Salomon , R. and Van Hemmen , J. L. 1996 . Accelerating backpropagation through dynamic self adaptation . Neural Netw. , 9 ( 4 ) : 589 – 601 .
   Web of Science ®Google Scholar
• Sigillito , V. G. , Wing , S. P. , Hutton , L. V. and Baker , K. B. 1989 . Classification of radar returns from the ionosphere using neural networks . Johns Hopkins APL Tech. Dig. , 10 : 262 – 266 .
   Web of Science ®Google Scholar
• Soucek , B. 1992 . Dynamic, Genetic, and Chaotic Programming , USA : John Wiley and Sons .
   Google Scholar
• Wang , X. G. , Tang , Z. , Tamura , M. and Ishii , M. 2004 . A modified error function for the backpropagation algorithm . Neurocomputing , 57 : 477 – 484 .
   Web of Science ®Google Scholar