Active inference and epistemic value

References

• Andreopoulos, A., & Tsotsos, J. (2013). A computational learning theory of active object recognition under uncertainty. International Journal of Computer Vision, 101(1), 95–142. doi:10.1007/s11263-012-0551-6
   Web of Science ®Google Scholar
• Atkeson, C., & Santamar ́ıa, J. (1997). A comparison of direct and model-based reinforcement learning. In International Conference on Robotics and Automation (ICRA). IEEE Press.
   Google Scholar
• Attias, H. (2003). Planning by probabilistic inference. In C. M. Bishop and B. J. Frey (Eds.), Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics. Key West, FL.
   Google Scholar
• Bach, D. R., & Dolan, R. J. (2012). Knowing how much you don’t know: A neural organization of uncertainty estimates. Nature Reviews. Neuroscience, 13(8), 572–586.
   PubMed Web of Science ®Google Scholar
• Baldassarre, G., & Mirolli, M. (2013). Intrinsically motivated learning in natural and artificial systems. Springer: Berlin.
   Google Scholar
• Ballard, D. H., Kit, D., Rothkopf, C. A., & Sullivan, B. (2013). A hierarchical modular architecture for embodied cognition. Multisensory Research, 26, 177. doi:10.1163/22134808-00002414
   Web of Science ®Google Scholar
• Balleine, B. W., & Dickinson, A. (1998). Goal-directed instrumental action: Contingency and incentive learning and their cortical substrates. Neuropharmacology, 37(4–5), 407–419. doi:10.1016/S0028-3908(98)00033-1
   Web of Science ®Google Scholar
• Barlow, H. (1961). Possible principles underlying the transformations of sensory messages. In W. Rosenblith (Ed.), Sensory communication (pp. 217–234). Cambridge, MA: MIT Press.
   Google Scholar
• Barlow, H. B. (1974). Inductive inference, coding, perception, and language. Perception, 3, 123–134. doi:10.1068/p030123
   Google Scholar
• Barto, A., Singh, S., & Chentanez, N. (2004). Intrinsically motivated learning of hierarchical collections of skills. In Proceedings of the 3rd International Conference on Development and Learning (ICDL 2004), San Diego: Salk Institute.
   Google Scholar
• Beal, M. J. (2003). Variational algorithms for approximate bayesian inference (PhD. Thesis). University College London.
   Google Scholar
• Beer, R. D. (1995). A dynamical systems perspective on agent-environment interaction. Artificial Intelligence, 72(1–2), 173–215. doi:10.1016/0004-3702(94)00005-L
   Web of Science ®Google Scholar
• Berridge, K. C. (2007). The debate over dopamine’s role in reward: The case for incentive salience. Psychopharmacology (Berl.), 191(3), 391–431. doi:10.1007/s00213-006-0578-x
   PubMed Web of Science ®Google Scholar
• Bialek, W., Nemenman, I., & Tishby, N. (2001). Predictability, complexity, and learning. Neural Computation, 13(11), 2409–2463. doi:10.1080/01621459.1989.10478825
   PubMed Web of Science ®Google Scholar
• Bonet, B., & Geffner, H. (2014). Belief tracking for planning with sensing: Width, complexity and approximations. Journal of Artificial Intelligence Research, 50, 923–970.
   Web of Science ®Google Scholar
• Botvinick, M., & An, J. (2008). Goal-directed decision making in prefrontal cortex: A computational framework. Advances in Neural Information Processing Systems (NIPS).
   Google Scholar
• Botvinick, M., & Toussaint, M. (2012). Planning as inference. Trends in Cognitive Sciences, 16(10), 485–488. doi:10.1016/j.tics.2012.08.006
   PubMed Web of Science ®Google Scholar
• Braun, D. A., Ortega, P. A., Theodorou, E., & Schaal, S. (2011). Path integral control and bounded rationality. Adaptive Dynamic Programming And Reinforcement Learning (ADPRL), 2011 IEEE Symposium on, Paris, IEEE.
   Google Scholar
• Bromberg-Martin, E. S., & Hikosaka, O. (2009). Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron, 63(1), 119–126. doi:10.1016/j.neuron.2009.06.009
   Google Scholar
• Brooks, R. (1991). Intelligence without representation. Artificial Intelligence, 47(1–3), 139–159. doi:10.1016/0004-3702(91)90053-M
   Web of Science ®Google Scholar
• Bruce, N., & Tsotsos, J. (2009). Saliency, attention, and visual search: An information theoretic approach. Journal of Vision, 9 (3):5, 1–24. doi:10.1167/9.3.5
   PubMed Web of Science ®Google Scholar
• Bunzeck, N., & Düzel, E. (2006). Absolute coding of stimulus novelty in the human substantia nigra/VTA. Neuron, 51(3), 369–379. doi:10.1016/j.neuron.2006.06.021
   PubMed Web of Science ®Google Scholar
• Canolty, R. T., Edwards, E., Dalal, S. S., Soltani, M., Nagarajan, S. S., Kirsch, H. E. … Knight, R. T. (2006). High gamma power is phase-locked to theta oscillations in human neocortex. Science, 313(5793), 1626–1628. doi:10.1126/science.1128115
   PubMed Web of Science ®Google Scholar
• Cao, F., & Ray, S. (2012). Bayesian hierarchical reinforcement learning. Advances in Neural Information Processing Systems.
   Google Scholar
• Cisek, P. (2007). Cortical mechanisms of action selection: The affordance competition hypothesis. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 362(1485), 1585–1599.
   PubMed Web of Science ®Google Scholar
• Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. The Behavioral and Brain Sciences, 36(3), 181–204. doi:10.1017/S0140525X12000477
   PubMed Web of Science ®Google Scholar
• Cohen, J. D., McClure, S. M., & Yu, A. J. (2007). Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 933–942. doi:10.1098/rstb.2007.2098
   PubMed Web of Science ®Google Scholar
• D’Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008). BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science, 319(5867), 1264–1267. doi:10.1126/science.1150605
   PubMed Web of Science ®Google Scholar
• Daunizeau, J., Den Ouden, H. E., Pessiglione, M., Kiebel, S. J., Stephan, K. E., & Friston, K. J. (2010). Observing the observer (I): Meta-bayesian models of learning and decision-making. PLoS One, 5(12), e15554. doi:10.1371/journal.pone.0015554
   Google Scholar
• Daw, N. D., & Doya, K. (2006). The computational neurobiology of learning and reward. Current Opinion in Neurobiology, 16(2), 199–204. doi:10.1016/j.conb.2006.03.006
   PubMed Web of Science ®Google Scholar
• Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8(12), 1704–1711. doi:10.1038/nn1560
   PubMed Web of Science ®Google Scholar
• Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095), 876–879. doi:10.1038/nature04766
   PubMed Web of Science ®Google Scholar
• Dayan, P. (2009). Dopamine, reinforcement learning, and addiction. Pharmacopsychiatry, 42(1), S56–65. doi:10.1055/s-0028-1124107
   Google Scholar
• Dayan, P., & Hinton, G. E. (1997). Using expectation maximization for reinforcement learning. Neural Computation, 9, 271–278. doi:10.1162/neco.1997.9.2.271
   Web of Science ®Google Scholar
• Dayan, P., Hinton, G. E., & Neal, R. (1995). The Helmholtz machine. Neural Computation, 7, 889–904. doi:10.1162/neco.1995.7.5.889
   PubMed Web of Science ®Google Scholar
• De Martino, B., Fleming, S. M., Garrett, N., & Dolan, R. J. (2012). Confidence in value-based choice. Nature Neuroscience, 16, 105–110. doi:10.1038/nn.3279
   PubMed Web of Science ®Google Scholar
• Dolan, R. J., & Dayan, P. (2013). Goals and habits in the brain. Neuron, 80(2), 312–325. doi:10.1016/j.neuron.2013.09.007
   PubMed Web of Science ®Google Scholar
• Ferro, M., Ognibene, D., Pezzulo, G., & Pirrelli, V. (2010). Reading as active sensing: A computational model of gaze planning during word recognition. Frontiers in Neurorobotics, 4, 1.
   Google Scholar
• Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299(5614), 1898–1902. doi:10.1126/science.1077349
   PubMed Web of Science ®Google Scholar
• FitzGerald, T., Dolan, R., & Friston, K. (2014). Model averaging, optimal inference, and habit formation. Frontiers in Human Neuroscience. doi:10.3389/fnhum.2014.00457
   Google Scholar
• Fox, C., & Roberts, S. (2011). A tutorial on variational Bayes. Artificial Intelligence Review, Springer. doi:10.1007/s10462-10011-19236-10468
   Google Scholar
• Friston, K. (2010). The free-energy principle: A unified brain theory? Nature Reviews. Neuroscience, 11(2), 127–138. doi:10.1038/nrn2787
   PubMed Web of Science ®Google Scholar
• Friston, K., Adams, R. A., Perrinet, L., & Breakspear, M. (2012). Perceptions as hypotheses: Saccades as experiments. Frontiers in Psychology, 3, 151. doi:10.3389/fpsyg.2012.00151
   PubMed Web of Science ®Google Scholar
• Friston, K., & Mathys, C. (2015). I think therefore I am. Cognitive Dynamic Systems. S. Haykin, IEEE press: in press.
   Google Scholar
• Friston, K., Samothrakis, S., & Montague, R. (2012). Active inference and agency: Optimal control without cost functions. Biological Cybernetics, 106, 523–541. [Epub ahead of print]. doi:10.1007/s00422-012-0512-8
   PubMed Web of Science ®Google Scholar
• Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2014). The anatomy of choice: Dopamine and decision-making. Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1655). doi:10.1098/rstb.2013.0481
   PubMed Web of Science ®Google Scholar
• Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., Raymond, R. J., & Dolan, J. (2013). The anatomy of choice: Active inference and agency. Frontiers in Human Neuroscience, 7, 598. doi:10.3389/fnhum.2013.00598
   PubMed Web of Science ®Google Scholar
• Furmston, T., & Barber, D. (2010). Variational methods for reinforcement learning. In International Conference on Artificial Intelligence and Statistics (AISTATS). JMLR: W&CP, vol. 9, (pp. 241–248).
   Google Scholar
• Fuster, J. M. (2004). Upper processing stages of the perception-action cycle. Trends in Cognitive Sciences, 8(4), 143–145. doi:10.1016/j.tics.2004.02.004
   PubMed Web of Science ®Google Scholar
• Gurney, K., Prescott, T. J., & Redgrave, P. (2001). A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biological Cybernetics, 84(6), 401–410. doi:10.1007/PL00007984
   PubMed Web of Science ®Google Scholar
• Harlow, H. (1950). Learning and satiation of response to intrinisically motivated complex puzzle performance by monkeys.” Journal of Comparative Physiological Psychology, 40, 289–294. doi:10.1037/h0058114
   Web of Science ®Google Scholar
• Hauskrecht, M. (2000). Value-function approximations for partially observable Markov decision processes. Journal of Artificial Intelligence Research, 13, 33–94.
   Web of Science ®Google Scholar
• Helmholtz, H. (1866/1962). Concerning the perceptions in general. Treatise on physiological optics. New York, NY: Dover. III.
   Google Scholar
• Hollerman, J. R., & Schultz, W. (1996). Activity of dopamine neurons during learning in a familiar task context. Social Neuroscience Abstracts, 22, 1388.
   Google Scholar
• Howard, R. (1966). Information value theory. IEEE Transactions on Systems, Science and Cybernetics, SSC-2 (1),22–26. doi:10.1109/TSSC.1966.300074
   Google Scholar
• Humphries, M. D., Khamassi, M., & Gurney, K. (2012). Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia. Frontiers in Neuroscience, 6, 9. doi:10.3389/fnins.2012.00009
   PubMed Web of Science ®Google Scholar
• Itti, L., & Baldi, P. (2009). Bayesian surprise attracts human attention. Vision Research, 49(10), 1295–1306. doi:10.1016/j.visres.2008.09.007
   PubMed Web of Science ®Google Scholar
• Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nature Reviews. Neuroscience, 2(3), 194–203. doi:10.1038/35058500
   PubMed Web of Science ®Google Scholar
• Jezek, K., Henriksen, E., Treves, A., Moser, E., & Moser, M.-B. (2011). Theta-paced flickering between place-cell maps in the hippocampus. Nature, 478, 246–249. doi:10.1038/nature10439
   PubMed Web of Science ®Google Scholar
• Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1–2), 99–134. doi:10.1016/S0004-3702(98)00023-X
   Web of Science ®Google Scholar
• Kakade, S., & Dayan, P. (2002). Dopamine: Generalization and bonuses. Neural Networks, 15(4–6), 549–559. doi:10.1016/S0893-6080(02)00048-5
   Web of Science ®Google Scholar
• Kamar., E., & Horvitz, E. (2013). Light at the end of the tunnel: A Monte Carlo approach to computing value of information. In Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems.
   Google Scholar
• Kappen, H. J., Gomez, Y., & Opper, M. (2012). Optimal control as a graphical model inference problem. Machine Learning, 87(2), 159–182. doi:10.1007/s10994-012-5278-7
   Web of Science ®Google Scholar
• Kass, R. E., & Steffey, D. (1989). Approximate Bayesian inference in conditionally independent hierarchical models (parametric empirical Bayes models). Journal of the American Statistical Association, 407, 717–726.
   Google Scholar
• Kirsh, D., & Maglio, P. (1994). On distinguishing epistemic from pragmatic action. Cognitive Science, 18(4), 513–549. doi:10.1207/s15516709cog1804_1
   Web of Science ®Google Scholar
• Klyubin, A. S., Polani, D., & Nehaniv, C. L. (2008). Keep your options open: An information-based driving principle for sensorimotor systems. PLoS One, 3(12), e4018. doi:10.1371/journal.pone.0004018
   Google Scholar
• Krause., A., & Guestrin, C. (2005). Optimal nonmyopic value of information in graphical models: Efficient algorithms and theoretical limits. In International Joint Conference on Artificial Intelligence (IJCAI). Carnegie Mellon University.
   Google Scholar
• Krebs, R. M., Schott, B. H., Schütze, H., & Düzel, E. (2009). The novelty exploration bonus and its attentional modulation. Neuropsychologia, 47, 2272–2281. doi:10.1016/j.neuropsychologia.2009.01.015
   PubMed Web of Science ®Google Scholar
• Lee, T. S., & Mumford, D. (2003). Hierarchical Bayesian inference in the visual cortex. Journal of the Optical Society of America A, 20, 1434–1448. doi:10.1364/JOSAA.20.001434
   Google Scholar
• Lepora, N., Martinez-Hernandez, U., & Prescott, T. (2013). Active touch for robust perception under position uncertainty. In IEEE proceedings of ICRA.
   Google Scholar
• Linsker, R. (1990). Perceptual neural organization: Some approaches based on network models and information theory. Annual Review of Neuroscience, 13, 257–281. doi:10.1146/annurev.ne.13.030190.001353
   PubMed Web of Science ®Google Scholar
• Little, D. Y., & Sommer, F. T. (2013). Learning and exploration in action-perception loops. Front Neural Circuits, 7, 37. doi:10.3389/fncir.2013.00037
   PubMed Web of Science ®Google Scholar
• Littman, M., Sutton, R., & Singh, S. (2002). Predictive Representations of State. Advances in Neural Information Processing Systems 14 (NIPS).
   Google Scholar
• Lungarella, M., & Sporns, O. (2006). Mapping information flow in sensorimotor networks. PLoS Computational Biology, 2, e144. doi:10.1371/journal.pcbi.0020144
   Google Scholar
• McClure, S. M., Daw, N. D., & Montague, P. R. (2003). A computational substrate for incentive salience. Trends in Neurosciences, 26(8), 423–428. doi:10.1016/S0166-2236(03)00177-2
   PubMed Web of Science ®Google Scholar
• Miller, G., Galanter, E., & Pribram, K. (1960). Plans and the structure of behavio. New York, NY: Henry Holt.
   Google Scholar
• Moser, E. I., Kropff, E., & Moser, M.-B. (2008). Place cells, grid cells, and the brain’s spatial representation system. Annual Review of Neuroscience, 31, 69–89. doi:10.1146/annurev.neuro.31.061307.090723
   PubMed Web of Science ®Google Scholar
• Mushiake, H., Saito, N., Sakamoto, K., Itoyama, Y., & Tanji, J. (2006). Activity in the lateral prefrontal cortex reflects multiple steps of future events in action plans. Neuron, 50, 631–641. doi:10.1016/j.neuron.2006.03.045
   PubMed Web of Science ®Google Scholar
• Najemnik, J., & Geisler, W. (2005). Optimal eye movement strategies in visual search. Nature, 434, 387–391. doi:10.1038/nature03390
   PubMed Web of Science ®Google Scholar
• Nepora, N., & Gurney, K. (2012). The basal ganglia optimize decision making over general perceptual hypotheses. Neural Computation, 24(11), 2924–2945. doi:10.1162/NECO_a_00360
   PubMed Web of Science ®Google Scholar
• Niv, Y. (2007). Cost, benefit, tonic, phasic: What do response rates tell us about dopamine and motivation? Annals of the New York Academy of Sciences, 1104, 357–376.
   PubMed Web of Science ®Google Scholar
• O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304(5669), 452–454. doi:10.1126/science.1094285
   PubMed Web of Science ®Google Scholar
• O’Regan, J., & Noë, A. (2001). A sensorimotor account of vision and visual consciousness. The Behavioral and Brain Sciences, 24, 939–973. doi:10.1017/S0140525X01000115
   PubMed Web of Science ®Google Scholar
• Ognibene, D., & Baldassarre, G. (2014). Ecological active vision: Four Bio-inspired principles to integrate bottom-up and adaptive top-down attention tested with a simple camera-arm robot. Autonomous Mental Development, IEEE Transactions on. IEEE.
   Google Scholar
• Ognibene, D., Chinellato, E., Sarabia, M., & Demiris, Y. (2013). Contextual action recognition and target localization with an active allocation of attention on a humanoid robot. Bioinspiration & Biomimetics, 8, 3. doi:10.1088/1748-3182/8/3/035002
   Web of Science ®Google Scholar
• Ognibene, D., & Demiris, Y. (2013). Toward active event perception. In International Joint conference of Artificial Intelligence. IJCIA: Beijing.
   Google Scholar
• Ognibene, D.,Pezzulo, G., & Baldassarre, G. (2010). Learning to look in different environments: An active-vision model which learns and readapts visual routines. In S. Doncieux (ed.), Proceedings of the 11th International Conference on Simulation of Adaptive Behaviour , SAB 2010, Paris - Clos Lucé, France, August 25–28, 2010. Proceedings. Springer Berlin Heidelberg.
   Google Scholar
• Ognibene, D., Catenacci Volpi, N., Pezzulo, G., & Baldassare, G. (2013). Learning epistemic actions in model-free memory-free reinforcement learning: Experiments with a neuro-robotic model. In Second International Conference on Biomimetic and Biohybrid Systems. Proceedings. pp. 191–203. doi:10.1007/978-3-642-39802-5_17
   Google Scholar
• Oja, E. (1989). Neural networks, principal components, and subspaces. International Journal of Neural Systems, 1, 61–68. doi:10.1142/S0129065789000475
   Google Scholar
• Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607–609. doi:10.1038/381607a0
   PubMed Web of Science ®Google Scholar
• Optican, L., & Richmond, B. J. (1987). Temporal encoding of two-dimensional patterns by single units in primate inferior cortex. II Information Theoretic Analysis. Journal of Neurophysiology, 57, 132–146.
   PubMed Web of Science ®Google Scholar
• Ortega, P. A., & Braun, D. A. (2011). Information, utility and bounded rationality. Lecture Notes on Artificial Intelligence, 6830, 269–274.
   Google Scholar
• Ortega, P. A., & Braun, D. A. (2013). Thermodynamics as a theory of decision-making with information-processing costs. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 469, 2153. doi:10.1098/rspa.2012.0683
   Web of Science ®Google Scholar
• Oudeyer, P.-Y., & Kaplan, F. (2007). What is intrinsic motivation? A typology of computational approaches. Frontiers in Neurorobotics, 1, 6. doi:10.3389/neuro.12.006.2007
   PubMedGoogle Scholar
• Penny, W., Zeidman, P., & Burgess, N. (2013). Forward and backward inference in spatial cognition. Plos Computational Biology, 9(12), e1003383. doi:10.1371/journal.pcbi.1003383
   Google Scholar
• Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006). Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature, 442(7106), 1042–1045. doi:10.1038/nature05051
   PubMed Web of Science ®Google Scholar
• Pezzementi, Z., Plaku, E., Reyda, C., & Hager, G. (2011). Tactile-object recognition from appearance information. IEEE Transactions on Robotics, 27, 473–487. doi:10.1109/TRO.2011.2125350
   Web of Science ®Google Scholar
• Pezzulo, G., & Castelfranchi, C. (2009). Thinking as the control of imagination: A conceptual framework for goal-directed systems. Psychological Research Psychologische Forschung, 73, 559–577. doi:10.1007/s00426-009-0237-z
   PubMed Web of Science ®Google Scholar
• Pezzulo, G., Rigoli, F., & Chersi, F. (2013). The mixed instrumental controller: Using value of information to combine habitual choice and mental simulation. Frontiers in Psychology, 4, 92. doi:10.3389/fpsyg.2013.00092
   Google Scholar
• Pezzulo, G., Van Der Meer, M., Lansink, C., & Pennartz, C. (2014). Internally generated sequences in learning and executing goal-directed behavior. Trends in Cognitive Sciences, 18(2), pp. 647–657.
   Google Scholar
• Rao, R. P., & Ballard, D. H. (1999). Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1), 79–87. doi:10.1038/4580
   PubMed Web of Science ®Google Scholar
• Redgrave, P., & Gurney, K. (2006). The short-latency dopamine signal: A role in discovering novel actions? Nature Reviews. Neuroscience, 7(12), 967–975. doi:10.1038/nrn2022
   PubMed Web of Science ®Google Scholar
• Roy, N., Burgard, W., Fox, D., & Thrun, S. (1999). Coastal navigation: Robot navigation under uncertainty in dynamic environments. In Proceedings of the IEEE International Conference on Robotics andAutomation (ICRA). Washington, DC: IEEE Computer Society.
   Google Scholar
• Ryan, R., & Deci, E. (1985). Intrinsic motivation and self-determination in human behavior. New York, NY: Plenum.
   Google Scholar
• Schembri, M., Mirolli, M., & Baldassare, G. (2007). Evolving internal reinforcers for an intrinsically motivated reinforcement learning robot. In Y. Demiris, B. Scassellati, & D. Mareschal (Eds.) Proceedings of the 6th IEEE International Conference on Development and Learning (ICDL2007). IEEE.
   Google Scholar
• Schmidhuber, J. (1991). Curious model-building control systems. In Proceeding of International Joint Conference on Neural Networks. Singapore: IEEE. 2: 1458–1463.
   Google Scholar
• Schneider, A., Sturm, J., Stachniss, C., Reisert, M., Burkhardt, H., & Burgard, W. (2009). Object identification with tactile sensors using bag-of-features. IROS 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2009. p. 243. IEEE.
   Google Scholar
• Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80(1), 1–27.
   PubMed Web of Science ®Google Scholar
• Schultz, W., Apicella, P., & Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. Journal of Neuroscience, 13, 900–913.
   PubMed Web of Science ®Google Scholar
• Schwartenbeck, P., FitzGerald, T., Mathys, C., Dolan, R., & Friston, K. (2014). The dopaminergic midbrain encodes the expected certainty about desired outcomes. Cerebral Cortex. pii: bhu159. doi:10.1093/cercor/bhu159
   PubMedGoogle Scholar
• Shackle, G. (1972). Epistemics and economics. Cambridge, MA: Cambridge University Press.
   Google Scholar
• Singh, A., Krause, A., Guestrin, C., & Kaiser, W. (2009). Efficient informative sensing using multiple robots. Journal of Artificial Intelligence Research, 34(2), 707.
   Google Scholar
• Solway, A., & Botvinick, M. (2012). Goal-directed decision making as probabilistic inference: A com- putational framework and potential neural correlates. Psychological Review, 119, 120–154. doi:10.1037/a0026435
   PubMed Web of Science ®Google Scholar
• Sornkarn, N., Nanayakkara, T., & Howard, M. (2014). Internal impedance control helps information gain in embodied perception. In IEEE International Conference on Robotics and Automation (ICRA), IEEE.
   Google Scholar
• Still, S. (2009). Information-theoretic approach to interactive learning. EPL (Europhysics Letters), 85(2), 28005. doi:10.1209/0295-5075/85/28005
   Google Scholar
• Still, S., & Precup, D. (2012). An information-theoretic approach to curiosity-driven reinforcement learning. Theory in Biosciences, 131(3), 139–148. doi:10.1007/s12064-011-0142-z
   PubMed Web of Science ®Google Scholar
• Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
   Google Scholar
• Tani, J., & Nolfi, S. (1999). Learning to perceive the world as articulated: An approach for hierarchical learning in sensory–motor systems. Neural Networks, 12, 1131–1141. doi:10.1016/S0893-6080(99)00060-X
   PubMed Web of Science ®Google Scholar
• Tishby, N., & Polani, D. (2010). Information theory of decisions and actions. In V. Cutsuridis, A. Hussain, & J. Taylor (Eds.), Perception-reason-action cycle: Models, algorithms and systems. Springer: Berlin.
   Google Scholar
• Toussaint, M., & Storkey, A. (2006). Probabilistic inference for solving discrete and continuous state Markov Decision Processes. In Proceeding of the 23nd International Conference on Machine Learning. ACM. pp. 945–952.
   Google Scholar
• Van den Broek, J. L., Wiegerinck, W. A. J. J., & Kappen, H. J. (2010). Risk-sensitive path integral control. UAI, 6, 1–8.
   Google Scholar
• Verschure, P. F., Voegtlin, T., & Douglas, R. J. (2003). Environmentally mediated synergy between perception and behavior in mobile robots. Nature, 425, 620–624. doi:10.1038/nature02024
   PubMed Web of Science ®Google Scholar
• Vijayakumar, S., Toussaint, M., Petkos, G., & Howard, M. (2009). Planning and moving in dynamic environments. In B. Sendhoff (Ed.), Creating brain-like intelligence (pp. 151–191). Berlin: Springer-Verlag.
   Google Scholar
• Viswanathan, G., Buldyrev, S., Havlin, S., Da Luz, M., Raposo, E., & Stanley, H. (1999). Optimizing the success of random searches. Nature, 401(6756), 911–914. doi:10.1038/44831
   PubMed Web of Science ®Google Scholar
• Walther, D., Rutishauser, U., Koch, C., & Perona, P. (2005). Selective visual attention enables learning and recognition of multiple objects in cluttered scenes. Computer Vision and Image Understanding, 100, 41–63. doi:10.1016/j.cviu.2004.09.004
   Web of Science ®Google Scholar
• Wittmann, B. C., Daw, N. D., Seymour, B., & Dolan, R. J. (2008). Striatal activity underlies novelty-based choice in humans. Neuron, 58(6), 967–973. doi:10.1016/j.neuron.2008.04.027
   PubMed Web of Science ®Google Scholar