A learning rule that explains how rewards teach attention

REFERENCES

• Ahissar, M., & Hochstein, S. (1993). Attentional control of early perceptual learning. Proceedings of the National Academy of Sciences of the United States of America, 90, 5718–5722. doi:10.1073/pnas.90.12.5718
   PubMed Web of Science ®Google Scholar
• Anderson, B. A., Laurent, P. A., & Yantis, S. (2011). Value-driven attentional capture. Proceedings of the National Academy of Sciences of the United States of America, 108, 10367–10371. doi:10.1073/pnas.1104047108
   PubMed Web of Science ®Google Scholar
• Ashby, F. G., Ennis, J. M., & Spiering, B. J. (2007). A neurobiological theory of automaticity in perceptual categorization. Psychological Review, 114, 632–656. doi:10.1037/0033-295X.114.3.632
   PubMed Web of Science ®Google Scholar
• Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010). Dopamine in motivational control: Rewarding, aversive, and alerting. Neuron, 68, 815–834. doi:10.1016/j.neuron.2010.11.022
   PubMed Web of Science ®Google Scholar
• Bundesen, C., Habekost, T., & Kyllingsbæk, S. (2005). A Neural theory of visual attention: Bridging cognition and neurophysiology. Psychological Review, 112, 291–328. doi:10.1037/0033-295X.112.2.291
   PubMed Web of Science ®Google Scholar
• Chelazzi, L., Perlato, A., Santandrea, E., & Libera, D. C. (2013). Rewards teach visual selective attention. Vision Research, 85, 58–72. doi:10.1016/j.visres.2012.12.005
   PubMed Web of Science ®Google Scholar
• Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201–215. doi:10.1038/nrn755
   PubMed Web of Science ®Google Scholar
• Dayan, P., & Balleine, B. W. (2002). Reward, motivation, and reinforcement learning. Neuron, 36, 285–298. doi:10.1016/S0896-6273(02)00963-7
   PubMed Web of Science ®Google Scholar
• Dayan, P., & Yu, A. J. (2002). ACh, uncertainty, and cortical inference (Vol. 1, p. 189). Paper Presented at the Advances in Neural Information Processing Systems, MIT Press.
   Google Scholar
• Dehaene, S., Dehaene-Lambertz, G., & Cohen, L. (1998). Abstract representations of numbers in the animal and human brain. Trends in Neurosciences, 21, 355–361. doi:10.1016/S0166-2236(98)01263-6
   PubMed Web of Science ®Google Scholar
• Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18, 193–222. doi:10.1146/annurev.ne.18.030195.001205
   PubMed Web of Science ®Google Scholar
• Deubel, H., & Schneider, W. X. (1996). Saccade target selection and object recognition: Evidence for a common attentional mechanism. Vision Research, 36, 1827–1837. doi:10.1016/0042-6989(95)00294-4
   PubMed Web of Science ®Google Scholar
• Duncan, J. (2010). The multiple-demand (MD) system of the primate brain: Mental programs for intelligent behaviour. Trends in Cognitive Sciences, 14, 172–179. doi:10.1016/j.tics.2010.01.004
   PubMed Web of Science ®Google Scholar
• Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1(1), 1–47. doi:10.1093/cercor/1.1.1
   PubMed Web of Science ®Google Scholar
• Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. S. (1989). Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. Journal of Neurophysiology, 61, 331–349.
   PubMed Web of Science ®Google Scholar
• Gnadt, J. W., & Andersen, R. A. (1988). Memory related motor planning activity in posterior parietal cortex of macaque. Experimental Brain Research, 70, 216–220.
   PubMed Web of Science ®Google Scholar
• Gottlieb, J., & Balan, P. (2010). Attention as a decision in information space. Trends in Cognitive Sciences, 14, 240–248. doi:10.1016/j.tics.2010.03.001
   PubMed Web of Science ®Google Scholar
• Hickey, C., Chelazzi, L., & Theeuwes, J. (2010). Reward changes salience in human vision via the anterior cingulate. Journal of Neuroscience, 30, 11096–11103. doi:10.1523/JNEUROSCI.1026-10.2010
   PubMed Web of Science ®Google Scholar
• Jiang, Y., & Chun, M. M. (2001). Selective attention modulates implicit learning. The Quarterly Journal of Experimental Psychology A, 54, 1105–1124. doi:10.1080/02724980042000516
   PubMed Web of Science ®Google Scholar
• Kilgard, M. P., & Merzenich, M. M. (1998). Cortical map reorganization enabled by Nucleus Basalis Activity. Science, 279, 1714–1718. doi:10.1126/science.279.5357.1714
   PubMed Web of Science ®Google Scholar
• Kowler, E., Anderson, E., Dosher, B., & Blaser, E. (1995). The role of attention in the programming of saccades. Vision Research, 35, 1897–1916. doi:10.1016/0042-6989(94)00279-U
   PubMed Web of Science ®Google Scholar
• Krueger, K. A., & Dayan, P. (2009). Flexible shaping: How learning in small steps helps. Cognition, 110, 380–394. doi:10.1016/j.cognition.2008.11.014
   PubMed Web of Science ®Google Scholar
• Lennert, T., & Martinez-Trujillo, J. (2011). Strength of response suppression to distracter stimuli determines attentional-filtering performance in primate prefrontal neurons. Neuron, 70(1), 141–152. doi:10.1016/j.neuron.2011.02.041
   PubMed Web of Science ®Google Scholar
• Libera, D. C., & Chelazzi, L. (2009). Learning to attend and to ignore is a matter of gains and losses. Psychological Science, 20, 778–784. doi:10.1111/j.1467-9280.2009.02360.x
   PubMed Web of Science ®Google Scholar
• Liu, Z., Zhou, J., Li, Y., Hu, F., Lu, Y., Ma, M., … Luo, M. (2014). Dorsal raphe neurons signal reward through 5-HT and glutamate. Neuron, 81, 1360–1374. doi:10.1016/j.neuron.2014.02.010
   PubMed Web of Science ®Google Scholar
• Louie, K., Grattan, L. E., & Glimcher, P. W. (2011). Reward value-based gain control: Divisive normalization in parietal cortex. Journal of Neuroscience, 31, 10627–10639. doi:10.1523/JNEUROSCI.1237-11.2011
   PubMed Web of Science ®Google Scholar
• Mao, T., Kusefoglu, D., Hooks, B. M., Huber, D., Petreanu, L., & Svoboda, K. (2011). Long-Range neuronal circuits underlying the interaction between sensory and motor cortex. Neuron, 72(1), 111–123. doi:10.1016/j.neuron.2011.07.029
   PubMed Web of Science ®Google Scholar
• Maunsell, J. H. R. (2004). Neuronal representations of cognitive state: Reward or attention? Trends in Cognitive Sciences, 8, 261–265. doi:10.1016/j.tics.2004.04.003
   PubMed Web of Science ®Google Scholar
• Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167–202. doi:10.1146/annurev.neuro.24.1.167
   PubMed Web of Science ®Google Scholar
• Pastor-Bernier, A., & Cisek, P. (2011). Neural correlates of biased competition in premotor cortex. Journal of Neuroscience, 31, 7083–7088. doi:10.1523/JNEUROSCI.5681-10.2011
   PubMed Web of Science ®Google Scholar
• Peck, C. J., Jangraw, D. C., Suzuki, M., Efem, R., & Gottlieb, J. (2009). Reward modulates attention independently of action value in posterior parietal cortex. Journal of Neuroscience, 29, 11182–11191. doi:10.1523/JNEUROSCI.1929-09.2009
   PubMed Web of Science ®Google Scholar
• Platt, M. L., & Glimcher, P. W. (1999). Neural correlates of decision variables in parietal cortex. Nature, 400, 233–238. doi:10.1038/22268
   PubMed Web of Science ®Google Scholar
• Raymond, J. E., & O’Brien, J. L. (2009). Selective visual attention and motivation the consequences of value learning in an attentional blink task. Psychological Science, 20, 981–988. doi:10.1111/j.1467-9280.2009.02391.x
   PubMed Web of Science ®Google Scholar
• Reynolds, J. H., & Chelazzi, L. (2004). Attentional modulation of visual processing. Annual Review of Neuroscience, 27, 611–647. doi:10.1146/annurev.neuro.26.041002.131039
   PubMed Web of Science ®Google Scholar
• Roelfsema, P. R. (2006). Cortical algorithms for perceptual grouping. Annual Review of Neuroscience, 29, 203–227. doi:10.1146/annurev.neuro.29.051605.112939
   PubMed Web of Science ®Google Scholar
• Roelfsema, P. R., & van Ooyen, A. (2005). Attention-gated reinforcement learning of internal representations for classification. Neural Computation, 17, 2176–2214. doi:10.1038/375484a0
   PubMed Web of Science ®Google Scholar
• Roelfsema, P. R., van Ooyen, A., & Watanabe, T. (2010). Perceptual learning rules based on reinforcers and attention. Trends in Cognitive Sciences, 14(2), 64–71. doi:10.1016/j.tics.2009.11.005
   PubMed Web of Science ®Google Scholar
• Rombouts, J. O., Bohte, S. M., & Roelfsema, P. R. (2012). Neurally plausible reinforcement learning of working memory tasks (Vol. 25, pp. 1880–1888). Paper Presented at the Advances in Neural Information Processing Systems, Lake Tahoe (Nevada, US).
   Google Scholar
• Rombouts, J. O., Bohte, S. M., & Roelfsema, P. R. (in press). How attention can create synaptic tags for the learning of working memories in sequential tasks. PLoS Computational Biology.
   Google Scholar
• Rummery, G. A., & Niranjan, M. (1994). On-line Q-learning using connectionist systems ( No. CUED/F-INFENG/TR 166). Cambridge: Cambridge Univsersity.
   Google Scholar
• Schultz, W. (2002). Getting formal with dopamine and reward. Neuron, 36, 241–263. doi:10.1016/S0896-6273(02)00967-4
   PubMed Web of Science ®Google Scholar
• Serences, J. T. (2008). Value-based modulations in human visual cortex. Neuron, 60, 1169–1181. doi:10.1016/j.neuron.2008.10.051
   PubMed Web of Science ®Google Scholar
• Stănişor, L., van der Togt, C., Pennartz, C. M., & Roelfsema, P. R. (2013). A unified selection signal for attention and reward in primary visual cortex. Proceedings of the National Academy of Sciences, 110, 9136–9141. doi:10.1073/pnas.1300117110/-/DCSupplemental
   PubMed Web of Science ®Google Scholar
• Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: A Bradford Book.
   Google Scholar
• Trabasso, T., & Bower, G. H. (1968). Attention in learning: Theory and research. New York, NY: Krieger.
   Google Scholar
• Treue, S., & Maunsell, J. H. (1996). Attentional modulation of visual motion processing in cortical areas MT and MST. Nature, 382, 539–541. doi:10.1038/382539a0
   PubMed Web of Science ®Google Scholar
• Whitehead, S. D., & Ballard, D. H. (1991). Learning to perceive and act by trial and error. Machine Learning, 7(1), 45–83.
   Web of Science ®Google Scholar