Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer

References

• Attneave F. Informational aspects of visual perception. Psychological Review 1954; 61: 183–93
   PubMed Web of Science ®Google Scholar
• Barlow H B. The coding of sensory messages. Current Problems in Animal Behaviour, W H Thorpe, O L Zangwill. Cambridge University Press. 1960; 331–60
   Google Scholar
• Shannon C E, Weaver W. The Mathematical Theory of Communication. University of Illinois Press, Urbana, IL 1949
   Google Scholar
• Blahut R E. Principles and Practice of Information Theory. Addison-Wesley, Cambridge, MA 1988
   Google Scholar
• Bialek W. Princeton Lectures on Biophysics, W Bialek. World Scientific, Singapore 1992
   Google Scholar
• Bialek W. Princeton Lectures on Biophysics, W Bialek. NEC, Princeton, NJ 1993
   Google Scholar
• Barlow H B, Foldiak P. Adaptation and decorrelation in the cortex. The Computing Neuron, R Durbin, C Miall, G Mitchison. Addison-Wesley, Cambridge, MA 1989; 54–72
   Google Scholar
• Atick J J. Could information theory provide an ecological theory of sensory processing?. Network 1992; 3: 213–51
   Web of Science ®Google Scholar
• Atick J J, Redlich A. What does the retina know about natural scenes?. Neural Comput. 1992; 4: 196–210
   Web of Science ®Google Scholar
• Atick J J, Li Z, Redlich A. Understanding retina color coding from first principles. Neural Comput. 1992; 4: 559–72
   Web of Science ®Google Scholar
• Li Z, Atick J J. Towards a theory of the striate cortex. Neural Comput. 1994; 6: 127–46
   Web of Science ®Google Scholar
• Li Z, Atick J J. Efficient stereo coding in the multiscale representation. Neural Comput. 1993, to appear
   Web of Science ®Google Scholar
• Laughlin S B. A simple coding procedure enhances a neuron's information capacity. Z. Naturf. 1981; 36: 910–2
   Google Scholar
• van Hateren J H. Theoretical predictions of spatio-temporal receptive fields of fly LMCs, and experimental validation. J. Comp. Physiology A 1992; 171: 157–70
   Web of Science ®Google Scholar
• Linsker R. Self-organization in a perceptual network. Computer 1988; 21: 105–17
   Web of Science ®Google Scholar
• Linsker R. Deriving receptive fields using an optimal encoding criterion. Neural Information Processing Systems, S J Hanson, J D Cowan, C Lee Giles. Morgan Kaufmann, San Mateo, CA 1993; 5: 953–60
   Google Scholar
• Bialek W, Rieke F, de Ruyter van Steveninck R, Warland D. Reading a neural code. Science 1991; 252: 1854–7
   PubMed Web of Science ®Google Scholar
• Nadal J-P, Parga N. Duality between learning machines: a bridge between supervised and unsupervised learning. Neural Comput. 1994; 6: 489–506
   Google Scholar
• Linsker R. Sensory processing and information theory. From Statistical Physics To Statistical Inference and Back, P Grassberger, J-P Nadal. Kluwer, Dordrecht 1994; 237–47
   Google Scholar
• Ruderman D. Designing receptive fields for highest fidelity. Network 1994; 5: 147–55
   Web of Science ®Google Scholar
• Chapeau-Blondeau F. Information entropy maximization in the transmission by a neuron nonlinearity. C. R. Acad. Sci. 1994, to appear
   Google Scholar
• Barlow H B, Kaushal T P, Mitchison G J. Finding minimum entropy codes. Neural Comput. 1989; 1: 412–23
   Web of Science ®Google Scholar
• Redlich A N. Redundancy reduction as a strategy for unsupervised learning. Neural Comput. 1993; 5: 289–304
   Web of Science ®Google Scholar
• Bialek W, Zee A. Understanding the efficiency of human perception. Phys. Rev. Lett. 1988; 61: 1512–5
   PubMed Web of Science ®Google Scholar
• Nadal J-P, Parga N. Information processing by a perceptron in an unsupervised learning task. Network 1993; 4: 295–312
   Web of Science ®Google Scholar
• Schuster H G. Learning by maximizing the information transfer through nonlinear noisy neurons and noise breakdown. Phys. Rev. A 1992; 46: 2131–8
   PubMed Web of Science ®Google Scholar
• Russ J C. The Image Processing Handbook. CRC Press, Boca Raton, FL 1992
   Google Scholar
• Barlow H B. Possible principles underlying the transformation of sensory messages. Sensory Communication, W Rosenblith. MIT Press, Cambridge, MA 1961; 217
   Google Scholar
• Nadal J-P, Parga N. Infomax with constrained variances: from noise reduction to decorrelation. 1994, in preparation
   Google Scholar
• Atick J J, Redlich A. Towards a theory of early visual processing. Neural Comput. 1990; 2: 308–20
   Web of Science ®Google Scholar
• Field D. Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. 1987; 4: 2379
   Web of Science ®Google Scholar
• Ruderman D, Bialek W. Statistics of natural images: scaling in the woods. Neural Information Processing Systems, J D Cowan, G Tesauro G, J Alspector. Morgan Kaufmann, San Mateo, CA 1994; 6
   Google Scholar
• Jutten C, Herault J. Signal Proc. 1991; 24: 1–10
   Web of Science ®Google Scholar
• Hopfield J J. Proc. Natl Acad. Sci. USA 1991; 88: 6462–66
   PubMed Web of Science ®Google Scholar
• Burel G. Blind separation of sources: a nonlinear neural algorithm. Neural Networks 1992; 5: 937–47
   Web of Science ®Google Scholar
• Linsker R. Local synaptic learning rules suffice to maximize mutual information in a linear network. Neural Comput. 1992; 4: 691–702
   Web of Science ®Google Scholar
• Atick J J, Redlich A N. Convergent algorithm for sensory receptive field development. Neural Comput. 1993; 5: 45–60
   Web of Science ®Google Scholar
• Rospars J-P, Fort J-C. Coding of odour quality: roles of convergence and inhibition. Network 1994; 5: 121–145
   Web of Science ®Google Scholar