Attention control learning in the decision space using state estimation

Abstract

The main goal of this paper is modelling attention while using it in efficient path planning of mobile robots. The key challenge in concurrently aiming these two goals is how to make an optimal, or near-optimal, decision in spite of time and processing power limitations, which inherently exist in a typical multi-sensor real-world robotic application. To efficiently recognise the environment under these two limitations, attention of an intelligent agent is controlled by employing the reinforcement learning framework. We propose an estimation method using estimated mixture-of-experts task and attention learning in perceptual space. An agent learns how to employ its sensory resources, and when to stop observing, by estimating its perceptual space. In this paper, static estimation of the state space in a learning task problem, which is examined in the Webots^TM simulator, is performed. Simulation results show that a robot learns how to achieve an optimal policy with a controlled cost by estimating the state space instead of continually updating sensory information.

Keywords:

• attention control
• state estimation
• Bayesian reinforcement learning
• decision-making
• mixture-of-experts

Additional information

Notes on contributors

Zahra Gharaee

Zahra Gharaee received her BSc degree in electrical engineering majoring control in 2009 and MSc degree in mechatronics in 2012 from K.N. Toosi University of Technology, Tehran, Iran. In 2014, she started her PhD degree in cognitive science from the Department of Philosophy, Lund University, Lund, Sweden. She is currently working as a member of the European project called What You See Is What You Did (WYSIWYD) to give a humanoid robot (iCub) the ability to percept human's actions. Her research interests include cognitive science and cognitive robotics, neuro science intelligent systems, learning systems, and machine learning.

Alireza Fatehi

Alireza Fatehi received his BS degree from Isfahan University of Technology, in 1990, MS degree from Tehran University, Tehran, Iran, in 1995, and PhD degree from Tohoku University, Sendai, Japan, in 2001, all in electrical engineering. Fatehi is an associate professor of electrical engineering in K.N. Toosi University of Technology (KNTU), Iran. He is the director of the Advance Process Automation & Control (APAC) research group and a member of Industrial Control Center of Excellence in KNTU. He is currently a visiting professor in the Department of Chemical and Material Engineering, University of Alberta, Canada. His research interests include industrial control systems, process control systems, intelligent systems, multiple model controller, nonlinear predictive controller, nonlinear identification, fault detection, and soft sensor.

Maryam S. Mirian

Maryam S. Mirian received her BSc degree in hardware engineering and MSc and PhD degrees in machine learning and robotics from the University of Tehran, Tehran, Iran, in 1999, 2003, and 2010, respectively. In 2011, she joined the Department of ECE, University of Tehran, as an assistant professor in machine learning and robotics group. Her current research interests include machine learning (specifically reinforcement learning and ensemble learning), statistical pattern recognition, and decision support systems.

Majid Nili Ahmadabadi

Majid Nili Ahmadabadi was born in 1967 and received his BS degree from Sharif University of Technology of Iran in 1990. He received his MSc and PhD degrees in information sciences from the Graduate School of Information Science, Tohoku University, Japan, in 1994 and 1997, respectively. In 1997, he joined the Advanced Robotics Laboratory at Tohoku University. Later, he moved to the School of Electrical and Computer Engineering, College of Engineering, University of Tehran, where he is a professor and the head of the Robotics and AI Laboratory. Dr Ahmadabadi is the founder and the director of the Cognitive Robotics Laboratory as well. He is also a senior researcher at the School of Cognitive Sciences, Institute for Research in Fundamental Sciences (IPM), Iran. In the summers of 2005 and 2008, Dr Ahmadabadi was with the Autonomous System Laboratory at EPFL and ETHZ as an invited visiting professor. He was one of the distinguished lecturers selected by the IEEE Robotics and Automation Society for the years 2007–2009. Dr Ahmadabadi served as a member of the Engineering Board of Iranian National Science Foundation for the period 2005–2011. He is one of the founders of The Robotics Society of Iran in addition to The Iranian Society of Cognitive Sciences. Dr Ahmadabadi is a member of the board of directors of The Mechatronics Society of Iran and The Robotics Society of Iran. His main research interests are cognitive robotics and modelling cognitive systems, learning systems, distributed robotics, object manipulation, and mobile robots.