Advanced search
Publication Cover
Network: Computation in Neural Systems Volume 7, 1996 - Issue 3
Submit an article Journal homepage
3
Views
0
CrossRef citations to date
0
Altmetric
Original Article

Divergence measures based on entropy families: a tool for guiding the growth of neural networks

H M A Andree Department of Physics and Astronomy, Utrecht University, PO Box 80000, 3508 TA, Utrecht, The Netherlands
,
A W Lodder Department of Physics and Astronomy, Utrecht University, PO Box 80000, 3508 TA, Utrecht, The Netherlands
&
A Taal Department of Physics and Astronomy, Utrecht University, PO Box 80000, 3508 TA, Utrecht, The Netherlands
Pages 533-554 | Received 20 Mar 1995, Published online: 09 Jul 2009

References

  • Baum E B, Haussler D. What size net gives valid generalization?. Neural Comput. 1989; 1: 151–60
  • Bose N K, Garga A K. Neural network design using Voronoi diagrams. IEEE Trans. Neural Networks 1993; NN-4: 778–87
  • Kohonen T. Self-Organisation and Associative Memory. Springer, Berlin 1989
  • Fahlman S E, Lebiere C. The cascade-correlation learning architecture. Advances in Neural Information Processing Systems, D S Touretzky. Morgan Kaufmann, San Mateo, CA 1990; 2: 524–32
  • Mézard M, Nadal J-P. Learning in feedforward layered networks: the tiling algorithm. J. Phys. A: Math. Gen. 1989; 22: 2191–203
  • Nadal J-P. Study of a growth algorithm for a feedforward network. Int. J. Neural Syst. 1989; 1: 55–9
  • Sirat J A, Nadal J-P. Neural trees: a new tool for classification. Network: Comput. Neural Syst. 1990; 1: 423–38
  • Barkema G T, Andree H M. A, Taal A. The patch algorithm: fast design of binary feedforward neural networks. Network: Comput. Neural Syst. 1993; 4: 393–407
  • Bichsel M, Seitz P. Minimum class entropy: a maximum information approach to layered networks. Neural Networks 1989; 2: 133–41
  • Marchand M, Golea M. On learning simple neural concepts: from halfspace intersections to neural decision lists. Network: Comput. Neural Syst. 1993; 4: 67–85
  • Andree H M A, Lodder A W, Taal A. The construction of feed-forward neural networks by means of genetic algorithms. Proc. 4th Int. Workshop on Software Engineering, Artificial Intelligence and Expert Systems for High-Energy and Nuclear Physics (Pisa), B Denby, D Perret-Gallix. World Scientific, Singapore 1995
  • Montana D J, Davis L. Training feedforward neural networks using genetic algorithms. Proc. 11th Int. Joint Conf. on Artificial Intelligence Detro. N S Sridharan. Morgan Kaufmann, San Mateo, CA 1989; 762–7
  • Richard M D, Lippmann R P. Neural network classifiers estimate Bayesian a posteriori, probabilities. Neural Comput. 1991; 3: 461–83
  • Keibek S A J, Barkema G T, Andree H M A, Savenije M H F, Taal A. A fast partitioning algorithm and a comparison of binary feedforward neural networks. Europhys. Lett. 1992; 18: 555–9
  • Breiman L, Friedman J H, Olshen R A, Stone C J. Classification and Regression Trees. Wadsworth and Brooks, Pacific Grove, CA 1984
  • Watanabe S. Pattern recognition as a quest for minimum entropy. Pattern Recognition 1981; 13: 381–7
  • Aczél J, Daróczy Z. On Measures of Information and their Characterizations. Academic, New York 1975
  • Havrda J, Chárvat F. Quantification method of classification processes. Concept of structural alpha-entropy. Kybernetika (Prague) 1967; 3: 30–5
  • Burbea J, Rao C R. On the convexity of some divergence measures based on entropy functions. IEEE Trans. Inf. Theory 1982; 28: 489–95
  • Shore J E, Johnson R W. Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Trans. Inf. Theory 1980; 26: 26–37
  • Rényi A. Selected papers of Alfréd Rényi, P Turán. Akadémiai Kiadó, Budapest 1976
  • Vapnik V. Principles of risk minimization for learning theory. Advances in Neural Information Processing Systems, J E Moody, S J Hanson, R P Lippman. Morgan Kaufmann, San Mateo, CA 1992; 4: 831–8, (Morgan Kaufmann)
  • Beckenbach E F, Bellman R. Inequalities. Springer, Berlin 1965
  • Mitrinovic D S. Analytic Inequalities. Springer, Berlin 1970
  • Nilsson N J. The Mathematical Foundations of Learning Machines. Morgan Kaufmann, San Mateo, CA 1990
  • Press W H, Flannery B P, Teukolsky S A, Vetterling W T. Numerical Recipes in C: the Art of Scientific Computing. Cambridge University Press, Cambridge 1988

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Order Reprints Request Corporate Permissions

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

Request Academic Permissions

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.