Uncertainty and Learning

REFERENCES

• A Dickinson, Contemporary Animal Learning Theory, Cambridge, Cambridge University Press, 1980.
   Google Scholar
• N J Mackintosh, Conditioning and Associative Learning, Oxford: Oxford University Press, 1983.
   Google Scholar
• M A Gluck, E S Reifsnider & R F Thompson, Adaptive signal processing and the cerebellum: Models of classical conditioning and VOR adaptation. In MA Gluck & DE Rumelhart, editors, Neuroscience and Connectionist Theory, Developments in Connectionist Theory, 131–185, Hillsdale, NJ: Erlbaum, 1990.
   Google Scholar
• W Schultz, Predictive reward signal of dopamine neurons, Journal of Neurophysiology, vol 80, pp 1–27, 1998.
   PubMed Web of Science ®Google Scholar
• D Marr, Vision, New York, NY: WH Freeman, 1982.
   Google Scholar
• J R Anderson, The Adaptive Character of Thought, Hillsdale, NJ: Erlbaum, 1990.
   Google Scholar
• L J Kamin, Predictability, surprise, attention, and conditioning. In: Punishment and Aversive Behavior, edited by Campbell BA, and Church RM, New York: Appleton-Century-Crofts, pp 242–259, 1969.
   Google Scholar
• R A Rescorla & A R Wagner, A theory of Pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement, In AH Black & WF Prokasy, editors, Classical Conditioning II: Current Research Theory, pp 64–69, New York, Appleton-Century-Crofts, 1972.
   Google Scholar
• R S Sutton & A G Barto, Toward a modern theory of adaptive networks: Expectation and prediction, Psychological Review, vol 88, pp 135–170, 1981.
   PubMed Web of Science ®Google Scholar
• B Widrow & M E Hoff, Adaptive switching circuits, WESCON Convention Report, vol 4, pp 96–104, 1960.
   Google Scholar
• RS Sutton & A G Barto, Time-derivative models of Pavlovian conditioning. In: M Gabriel & JW Moore, editors, Learning and Computational Neuroscience, pp 497–537. Cambridge, MA, MIT Press, 1990.
   Google Scholar
• P R Montague, P Dayan & T J Sejnowski, A framework for mesencephalic dopamine systems based on predictive Hebbian learning, Journal of Neuroscience, vol 16, pp 1936–1947, 1996.
   PubMed Web of Science ®Google Scholar
• W Schultz, P Dayan, & PR Montague, A neural substrate of prediction and reward. Science, vol 275, pp 1593–1599, 1997
   PubMed Web of Science ®Google Scholar
• W Schultz & A Dickinson, Neuronal coding of prediction errors, Annual Review of Neuroscience, vol 23, pp 473–500, 2000.
   PubMed Web of Science ®Google Scholar
• J M Pearce & G Hall, A model for Pavlovian learning: Variation in the effectiveness of conditioned but not unconditioned stimuli, Psychological Review, vol 87, pp 532–552, 1980.
   PubMed Web of Science ®Google Scholar
• PC Holland, Excitation and inhibition in unblocking. Journal of Experimental Psychology: Animal Behavior Processes, vol 14, pp 261–279, 1988.
   PubMed Web of Science ®Google Scholar
• P C Holland, Brain mechanisms for changes in processing of conditioned stimuli in Pavlovian conditioning: Implications for behavior theory, Animal Learning & Behavior, vol 25, pp 373–399, 1997.
   Google Scholar
• P C Holland & M Gallagher, Amygdala circuitry in attentional and representational processes, Trends in Cognitive Sciences, vol 3, pp 65–73, 1999.
   PubMed Web of Science ®Google Scholar
• Q Gu, Neuromodulatory transmitter systems in the cortex and their role in cortical plasticity, Neuroscience, vol 111, pp 815–835, 2002.
   PubMed Web of Science ®Google Scholar
• K Doya, Metalearning and neuromodulation, Neural Networks, vol 15, pp 495–506, 2002.
   PubMed Web of Science ®Google Scholar
• M E Hasselmo, Neuromodulation and cortical function: modeling the physiological basis of behaviour, Behavioural Brain Research, vol 67, pp 1–27. 1995.
   PubMed Web of Science ®Google Scholar
• P Dayan & A J Yu, A Ch, Uncertainty, and cortical inference. In TG Dietterich, S Becker & Z Ghahramani, editors, Advances in Neural Information Processing Systems, 14, 2002.
   Google Scholar
• A J Yu & P Dayan, Acetylcholine in cortical inference. Neural Networks, vol 15, pp 719–732, 2002.
   PubMed Web of Science ®Google Scholar
• J Rajkowski, P Kubiak & G Aston-Jones, Locus Coeruleus activity in monkey: phasic and tonic changes are associated with altered vigilance, Brain Res Bull, vol 35, pp 607–616, 1994.
   PubMed Web of Science ®Google Scholar
• G Aston-Jones, J Rajkowski & J Cohen, Role of locus coeruleus in attention and behavioral flexibility. Biol Psychiatry, vol 46, pp 1309–1320, 1999.
   PubMed Web of Science ®Google Scholar
• S J Sara, Learning by neurons: role of attention, reinforcement and behavior, Comptes Rendus de l'Academie des Sciences Serie III-Sciences de la Vie-Life Sciences, vol 321, pp 193–198, 1998.
   PubMedGoogle Scholar
• B D Anderson & J B Moore, Optimal Filtering, Prentice-Hall, Englewood Cliffs, NJ, 1979.
   Google Scholar
• P Dayan, S Kakade, & P R Montague, Learning and selective attention, Nature Neuroscience, vol 3, pp 1218–1223, 2000.
   PubMed Web of Science ®Google Scholar
• S Kakade & P Dayan, Acquisition and extinction in autoshaping, Psychological Review, vol 109, pp 533–544, 2002.
   PubMed Web of Science ®Google Scholar
• R S Sutton, Gain adaptation beats least squares? In Proceedings of the 7th Yale Workshop on Adaptive and learning Systems, 1992.
   Google Scholar
• N D Daw, S Kakade & P Dayan, Opponent interactions between serotonin and dopamine. Neural Networks, vol 15, pp 603–616, 2002.
   PubMed Web of Science ®Google Scholar
• D R Shanks, Forward and backward blocking in human contingency judgement. Quarterly Journal of Experimental Psychology: Comparative & Physiological Psychology, vol 37, pp 1–21, 1985.
   Web of Science ®Google Scholar
• R R Miller & H Matute, Biological significance in forward and backward blocking: Resolution of a discrepancy between animal conditioning and human causal judgement, Journal of Experimental Psychology: General, vol 125, pp 370–386, 1996.
   PubMed Web of Science ®Google Scholar
• P Dayan & S Kakade, Explaining away in weight space. In TK Leen, TG Dictterich & V Tresp, eds, Advances in Neural Information Processing Systems, 13, 2001, pp 451–457.
   Google Scholar
• A Yu & P Dayan, Expected and unexpected uncertainty: ACh and NE in the neocortex, Advances in Neural Information Processing Systems, vol 15, in press.
   Google Scholar
• M Gluck, P Glauthicr & R S Sutton, Adaptation of cue-specific learning rates in network models of human category learning, Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society, pp 540–545, Erlbaum, 1992.
   Google Scholar
• G E Hinton & T J Sejnowski, Unsupervised Learning: Foundations of Neural Computation, MIT Press, Cambridge, MA, 1999.
   Google Scholar
• D J Bucci, P C Holland & M Gallagher, Removal of cholinergic input to rat posterior parietal cortex disrupts incremental processing of conditioned stimuli, Journal of Neuroscience, vol 18, no 19, pp 8038–8046, October 1998.
   PubMed Web of Science ®Google Scholar
• J-S Han, P C Holland & M Gallagher, Disconnection of the amygdala central nucleus and substantia innominata/nucleus basalis disrupts increments in conditioned stimulus processing in rats, Behavioral Neuroscience, vol 113, pp 143–151, 1999.
   PubMed Web of Science ®Google Scholar
• D D Rasmusson, The role of acetylcholine in cortical synaptic plasticity, Behavioral Brain Research, vol 115, pp 205–218, November 2000.
   PubMed Web of Science ®Google Scholar
• A Dickinson & B M Balleine, The role of learning in motivation. In CR Gallistel (Editor), learning, Motivation & Emotion, Volume 3 of Steven's Handbook of Experimental Psychology, Third Edition, New York, John Wiley & Sons, 2002.
   Google Scholar
• P Dayan & B M Balleine, BW, Reward, Motivation and reinforcement learning, Neuron, in press.
   Google Scholar
• C R Gallistel & J Gibbon, Time, Rate, and Conditioning, Psychological Review, vol 107, pp 289–344, 2000.
   PubMed Web of Science ®Google Scholar
• P Dayan & T Long, Statistical models of conditioning. In MI Jordan, M Kearns & SA Solla, editors, Advances in Neural Information Processing Systems. Cambridge, MA, MIT Press, pp 117–123, 1998.
   Google Scholar
• J K Kruschke, Toward a unified model of attention in associative learning, Journal of Mathematical Psychology, vol 45, pp 812–863, 2001.
   Web of Science ®Google Scholar
• M Gluck & C Myers, Gateway to Memory: An Introduction to Neural Network Modeling of the Hippocampus and Learning, Cambridge, MA: MIT Press, 2000.
   Google Scholar
• Z Gil, B M Conners & Y Amitai, Differential regulation of neocortical synapses by neuromodulators and activity, Neuron, vol 19, pp 679–86, 1997.
   PubMed Web of Science ®Google Scholar
• F Kimura, M Fukuada & T Tsumoto, Acetylcholine suppresses the spread of excitation in the visual cortex revealed by optical recording: possible differential effect depending on the source of input, European Journal of Neuroscience, vol 11, pp 3597–3609, 1999.
   PubMed Web of Science ®Google Scholar
• C Y Hsieh, S J Cruikshank & R Metherate, Differential modulation of auditory thalamocortical and intracortical synaptic transmission by cholinergic agonist. Brain Research, vol 880, pp 51–64, 2000.
   PubMed Web of Science ®Google Scholar
• M E Hasselmo & M Cekic, Suppression of synaptic transmission may allow combination of associative feedback and self-organizing feedforward connections in the neocortex, Behav Brain Res. vol 79, pp 153–161, 1996.
   PubMed Web of Science ®Google Scholar
• S J Sara, A Vankov & A Herve, Locus coeruleus-evoked responses in behaving rats: a clue to the role of noradrenaline in memory, Brain Res Bull, vol 35, pp 457–465, 1994.
   PubMed Web of Science ®Google Scholar
• A Vankov, A Herve-Minvielle & S J Sara, Response to novelty and its rapid habituation in locus coeruleus neurons of freely exploring rat, Eur Journal Neurosci, vol 109, pp 903–911, 1995.
   Google Scholar
• M E Hasselmo, C Linster, M Patil, D Ma & M Cekic, Noradrenergic suppression of synaptic transmission may influence cortical signal-to-noise ratio, Journal Neurophysiol, vol 77, pp 3326–3339, 1997.
   PubMed Web of Science ®Google Scholar
• M Kobayashi, K Imamura, T Suai, N Onoda, M Yamamoto, S Komai & Y Watanabe, Selective suppression of horizontal propagation in rat visual cortex by norepinephrine, European Journal of Neuroscience, vol 12, pp 264–272, 2000.
   PubMed Web of Science ®Google Scholar
• G A Carpenter & S Grossberg, editors, Pattern Recognition by Self-Organizing Neural Networks. Cambridge, MA, MIT Press, 1991.
   Google Scholar
• T W Robbins, Cortical noradrenaline, attention and arousal, Psychological Medicine, vol 14, pp 13–21, 1984.
   PubMed Web of Science ®Google Scholar
• J McGaughy & M Sarter, Lack of effects of lesions of the dorsal noradrenergic bundle on behavioral vigilance, Behav Neurosci, vol 111, no 3, pp 646–652, 1997.
   PubMed Web of Science ®Google Scholar