Learning in neural network memories

References

• Rosenblatt, F. Principles of Neurodynamics. Spartan, Washington, DC 1961. Minsky, M, Papert, S. PerceptronsMITCambridge, MA, 1969
   Google Scholar
• Rumelhart D E, McClelland J L. Parallel Distributed Processing: Explorataions in the Microstructure of Cognition, D E Rumelhart, J L McClelland. MIT Press, Cambridge, MA 1986; I and II
   Google Scholar
• McCulloch, W S, Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 1943; 5: 115–33. Little, W A. The existence of persistent states in the brain. Math. Biosci.. 1975. 19. 101–20
   Google Scholar
• Hopfield J J. Neural networks and physical systems with emergent selective computational abilities. Proc. Natl. Acad. Sci. USA 1982; 79: 2554–258
   PubMed Web of Science ®Google Scholar
• Amari, S, Maginu, K. Statistical neurodynamics of associative memory. Neural Networks 1988; 1: 63–73. Grossberg, S. Nonlinear neural networks: principles, mechanisms and architecures. Neural Networks. 1988. 1. 17–61, For reviews and other viewpoints see, for example:Kohonen T 1988 An introduction to neural computing Neural Networks 1 3–16; 1984 SelfOrganisation and Associative Memory (Berlin: Springer)Amit D 1989 Modelling Brain Function: The World of Attractor Neural Networks (Cambridge: Cam-bridge University Press)
   Web of Science ®Google Scholar
• Hebb D O. The Organisation of Behaviour: A Neuropsychological Theory. Wiley, New York 1949
   Google Scholar
• Forrest, B M. Content-addressability and learning in neural networks. J. Phys. A: Math. Gen. 1988; 21: 245–55. Kohring, G A. Dynamical interference between the attractors in a neural network. Europhys. Lett.. 1989, to be published; Opper, M, Kleinz, J, Kinzel, WBasins of attraction near the critical capacity for neural networks with constant stabilities. J. Phys. A: Math. Gen.. 1989. 22 L407–11Krätzschnar, J, Kohring, G ARetrieval dyanamics of neural networks constructed from local and nonlocal learning rules. Preprint Bonn. 1989
   Google Scholar
• Kepler, T B, Abbott, L F. Domains of attraction in neural networks. J. Physique 1988; 49: 1657–62. Krauth, W, Nadal, J-P, Mézard, M. The role of stability and symmetry in the dynamics of neural networks. J. Phys. A: Math. Gen.. 1988. 21. 2995–3011
   Google Scholar
• Gardner, E. Maximum storage capacity in neural networks. Europhys. Lett. 1987; 4: 481–5. Gardner, E. The space of interactions in neural network models. J. Phys. A: Math. Gen.. 1988. 21. 257–70. Gardner, E, Derrida, BOptimal storage properties of neural network models. J. Phys. A: Math. Gen.. 1988. 21. 271–84
   Google Scholar
• Cover, T M. Geometrical and statistical properties of systems of linear inequalities with application in pattern recognition. IEEE Trans. Electromagnet. Compat. 1965; EC-14: 326–34, Venkatesh S 1986 Epsilon capacity of a neural network Neural Networks for Computing (Proc. Conf.; Snowbird, Utah, 1986) (AIP Conf. Proc. 151) ed J S Denker (New York: American Institute of Physics) pp 440–5 Baldi P and Venkatesh S 1987 The number of stable points for spin-glass and neural networks of higher order Phys. Rec. Letr. 58 913–6
   Google Scholar
• Abbott L F, Kepler T B. Universality in the space of interactions for network models. J. Phys. A: Math. Gen. 1989; 22: 2031–8
   Google Scholar
• Amit, D J, Gutfreund, H, Sompolinsky, H. Storing infinite numbers of patterns in a spin glass model of neural networks. Phys. Rev. Lett.. 1985; 55: 1530–3. Amit, D J, Gutfreund, H, Sompolinsky, H. Spin glass models of neural networks. Phys. Rev. A. 1985. 32: 1007–18. Amit, D J, Gutfreund, H, Sompolinsky, HInformation storage in neural networks with low levels of activity. Phys. Rev. A 198735: 2293–303. Amit, D J, Gutfreund, H, Sompolinsky, HStatistical mechanics of neural networks near saturation. Ann. Phys., NY. 1987. 173. 30–67
   PubMed Web of Science ®Google Scholar
• Kohonen, T, Reuhkala, E, Mäkisara, K, Vainio, L. Associative recall of images. Biol. Cyber. 1976; 22: 159–68. Personnaz, L, Guyon, I, Dreyfus, G. Information storage and retrieval in spin-glass like neural networks. J. Physique. 1985. 46 L359–65Kanter, I, Sompolinsky, HAssociative recall of memories without errors. Phys. Rev. A380–92
   PubMed Web of Science ®Google Scholar
• Kelso, S R, Ganong, A H, Brown, T H. Hebbian synapses in the hippocampus. Proc. Natl. Acad. Sci. USA 1986; 83: 5326–30. diPrisco, G V. Hebb synaptic plasticity. Prog. Neurobiol.. 1984. 22. 89–102, Levy W B 1985 Associative changes at the synapse: LTP in the hippocampus Synaptic Modification, Neuron Seiecticity and Nervous System Organisation ed W B Levy, J A Anderson and S Lehmkuhle iHillsdale, NY: L Erlbaum Associates) pp 5–33Brown T H et a/ 1988 Long-term synaptic potentiation Science 242 724–8
   PubMed Web of Science ®Google Scholar
• Rauschecker, J P, Singer, J P. The effects of early visual experience on the cat's visual cortex and their possible explanation by Hebb synapses. J. Physiol. 1981; 310: 215–39, Singer W 1985 Hebbian modification of synaptic transmission as a common mechanism in experience-dependent maturation of cortical functions Synaptic Modification, Neuron Selectitvity and Nervous System Organisntion ed W B Levy, J A Anderson and S Lehmkuhle (Hillsdale, NY: L Erlbaum Associates) pp 35–64
   PubMed Web of Science ®Google Scholar
• Bear, M F, Cooper, L N, Ebner, F F. A physiological basis for a model of synapse modification. Science 1987; 237: 42–8. Reiter, H O, Stryker, M P. Neural plasticity without postsynaptic action potentials: less-active inputs become dominant when kitten visual cortex cells are pharmacologically inhibited. Proc. Natl. Acad. Sci. USA. 1988. 85. 3623–7
   PubMed Web of Science ®Google Scholar
• Peretto P. On learning and memory storage abilities of asymmetrical neural networks. J. Physique. 1988; 49: 711–26
   Google Scholar
• , This was suggested to me by T Kepler (private communication)
   Google Scholar
• Widrow B, Hoff M E. Adaptive switching circuits. WESCON Convention Report IV 1960; 96–104
   Google Scholar
• Diederich S, Opper M. Learning of correlated patterns in spin-glass networks by local learning rules. Phys. Rev. Lett. 1988; 58: 949–52
   Web of Science ®Google Scholar
• Carnahan B, Luther H A, Wilkes J O. Applied Numerical Methods. Wiley, New York 1969
   Google Scholar
• Berryman K W, Inchiosa M E, Jaffe A M, Janowsky S A. Convergence of an iterative neural network learning algorithm for linearly dependent patterns. Preprint. HUTMP 89/B237 Havard University. 1989
   Google Scholar
• Hertz J A, Thorbergson G I, Krogh A. Dynamics of learning in simple perceptrons. Phys. Scr. 1989; T25: 149–51
   Web of Science ®Google Scholar
• Kinzel W, Opper M. Dynamics of learning. Physics of Neural Networks, J L van Hemmen, E Domany, K Schulten. Springer, Berlin 1989, to appear
   Google Scholar
• Peretto P. On the dynamics of memorisation processes. Neural Networks 1988; 1: 309–22
   Web of Science ®Google Scholar
• Agmon, S. The relaxation method for linear inequalities. Can. J. Math. 1954; 6: 382–92. Jacobs, R. Increased rates of convergence through learning rate adaptation. Neural Networks. 1988. 1. 295–307
   Web of Science ®Google Scholar
• Krauth W, Mézard M. Learning algorithms with optimal stability for neural networks. J. Phys. A: Math. Gen. 1987; 20: L745–52
   Google Scholar
• Gardner, E, Stroud, N, Wallace, D J. Training with noise and the storage of correlated patterns in neural network models. Neural Computers, R Eckmiller, vander, C, Malsburg. Springer, Berlin 1988; 251–60, Pöppel G and Krey U 1987 Dynamical learning process for recognition of correlated patterns in symmetric spin glass models Europhys. Lett. 4 979–85
   Google Scholar
• Opper M. Learning times of neural networks: exact solution for a perceptron algorithm. Phys. Rev. A 1988; 38: 3824–6
   PubMed Web of Science ®Google Scholar
• Abbott L F, Kepler T B. Optimal learning in neural network memories. J. Phys. A: Math. Gen. 1989; 22: L711–7
   Google Scholar
• Parisi, G. A memory which forgets. J. Phys. A: Math. Gen. 1986; 19 L616–20van Hemmen, J L, Keller, G, K¨hn, R. Forgetful memories. Europhys. Lett.. 1988. 5: 663–8. Derrida, B, Nadal, J-PLearning and forgetting on asymmetric, diluted neural networks. J. Stat. Phys.. 1987. 49. 993–1009. Toulouse, G, Dehaene, S, Changeux, J-PSpin-glass model of learning by selection. Proc. Natl. Acad. Sci. USA. 1986. 83. 1695–8. Geszti, T, Pazmandi, FLearning within bounds and dream sleep. J. Phys. A: Math. Gen.. 1987. 20 L1299–303Gordon, M BMemory capacity of neural networks learning within bounds. J. Physique. 1987. 48. 2053–8. Nadal, J-P, Toulouse, G Changeux J-P, Dehaene, SNetworks of formal neurons and memory palimpsests. Europhys. Lett.. 1986. 1. 535–42
   Google Scholar
• Mézard M, Nadal J-P, Toulouse G. Solvable models of working memories. J. Physique 1986; 47: 1457–62
   Google Scholar
• Papadimitriou D, Steiglitz K. Combinatorical Optimisation: Algorithms and Complexity. NJ, Prentice Hall, Englewood Cliffs 1982
   Google Scholar
• Sompolinsky, H. Neural networks with nonlinear synapses and a static noise. Phys. Rev. A 1986; 34: 2571–4. van Hemmen, J L. Nonlinear neural networks near saturation. Phys. Rev. A. 1987. 36: 1959–62. Domany, E, Kinzel, W, Meir, RLayered neural networks. J. Phys. A: Math. Gen.. 1989. 22. 2081–102
   PubMed Web of Science ®Google Scholar
• Krauth, W, Opper, M. Critical storage capacity of the J=±1 neural network. J. Phys. A: Math. Gen. 1989; 22: L519–23. Krauth, W, Mézard, M. Storage capacity of memory networks with binary couplings. Preprint. 1989
   Google Scholar
• Dale H H. Pharmacology and nerve-endings. Proc. R. Soc. Med. 1935; 28: 319–32, Eccles J 1964 Physiology of Synapses (Berlin: Springer)Breitenberg V 1986 Brain Theory ed G Palm and A Aersten (Berlin: Springer)Peters A and Jones E G 1984 The Cerebral Cortex ed A Peters and E G Jones (New York: Plenum)
   PubMedGoogle Scholar
• Treves, A, Amit, D J. Metastable states in asymmetrical diluted Hopfield nets. J. Phys. A: Math. Gen. 1988; 21: 3155–69Low firing rates: an effective Hamiltonian for excitatory neurons. J. Phys. A: Math. Gen.. 1989. 22. 2205–26, Campbell C 1989 Neural network models with sign constrained weights Neural Computing ed J G Taylor and C Mannion (Bristol: Adam Hilger)Tsodyks M V and Feigel'man M V 1988 The enhanced storage capacity of neural networks with low activity level Europhys. Lett. 6 101–5Tsodyks M V 1988 Associative memory in asymmetric diluted network with low level of activity Europhys. Lett. 7 203–8Buhmann J, Divko V and Schulten K 1988 Associative memories with high information content Preprint Evans M R 1989 Random dilution in a neural network for biased patterns J. Phys. A: Math. Gen. 22 2103–18
   Google Scholar
• Shinomoto, S. A cognitive and associative memory. Biol. Cybern. 1987; 57: 197–206. Kohring, G A. Coexistence of global and local attractors in neural networks. Preprint. 1989, Bonn
   PubMed Web of Science ®Google Scholar
• Amit D J, Campbell C, Wong K Y M. The interaction space of neural networks with sign-constrained synapses. Preprint 1989
   Google Scholar
• Amit D J, Wong K Y M, Campbell C. Perceptron learning with sign-constrained weights. J. Phys. A: Math. Gen. 1989; 22: 2039–46
   Google Scholar
• Nadal, J-P, Toulouse, G. Information storage in sparsely-coded memory nets. Netw., Comput. Neural Syst. 1989; 1: 61–74. Rubin, N, Sompolinsky, H. Neural networks with low local firing rates. Europhys. Lett.. 1989. Golomb, D, Rubin, N, Sompolinsky, HThe Willshaw model: associative memory with sparse coding and low firing rates. Preprint. 1989, to appear
   Web of Science ®Google Scholar
• Willshaw D J, Buneman O P, Longuet-Higgins H C. Non-holographic associative memory. Nature 1969; 222: 960–2
   PubMed Web of Science ®Google Scholar
• Abeles M. Local Cortical Circuits. Springer, Berlin 1982, Miyashita Y and Chang H S 1988 Neuronal correlate of pictorial short-term memory in the primate temporal cortex Nature 331 68–70Fuster J M and Jervey J P 1982 Neuronal firing in the inferotemporal cortex of the monkey in a visual memory task J. Neurosci. 2 361–75
   Google Scholar
• Buhmann, J, Shulten, K. Noise driven temporal association in neural networks. Europhys. Lett. 1987; 4: 1205–9. Coolen, A C C, Gielen, C C A. M. Delays in neural networks. Europhys. Lett.. 1988. 7. 281–5, Abbott L F 1989 A network of oscillators Preprint Brandeis University
   Google Scholar
• Shinomoto, S. Memory maintenence in neural networks. J. Phys. A: Math. Gen. 1987; 20: L13059Hertz, J A, Thorbergson, G I, Krough, A. Phase transitions in simple learnings. J. Phys. A: Math. Gen.. 1989. 22. 2133–50. Meir, R, Domany, EIterated learning in a layered feed-forward neural network. Phys. Rev. A2660–8. Domany, E, Meir, R, Kinzel, WStoring and retrieving information in a layered spin system. Europhys. Lett.. 1986. 2. 175–85. Linsker, RFrom basic network principles to neural architecture: emergence of spatial-opponent cells. Proc. Natl. Acad. Sci. USA. 1986. 83. 7508–12From basic network principles to neural architecture: emergence of orientation-selection cells. Proc. Natl. Acad. Sci. USA. 1986. 83. 8390–4From basic network principles to neural architecture: emergence of orientation columns. Proc. Natl. Acad. Sci. USA. 1986. 83. 8779–83
   Google Scholar