Obtaining emergent behaviors for swarm robotics singling with deep reinforcement learning

ABSTRACT

Isolating (singling) an individual from a group can be essential for protection, rescue or capture tasks. In this paper a system with multiple shepherds who must coordinate the sheep to achieve a specific singleness is proposed. We present a realistically modeled system that will be finally tested in a real robotic system. We want to encourage the adaptability of the system and provide different solutions by promoting the emergence of the swarm. In this line we will focus on the use of reinforcement learning, avoiding a manual design of the behavior in order to not restrict the resulting behaviors and to facilitate their adaptation. A detailed MDP model will be specified as well as the keys to reduce its dimensionality and facilitate its training. We will check the results of the obtained singling policy with respect to a greedy policy and focus on evaluating different behavioral strategies that can solve the problem in different ways. In addition, one of the obtained policies will be analyzed in detail to check both its robustness and its scalability with respect to the number of shepherds and sheep. This policy will be finally tested on a physical robotic swarm.

GRAPHICAL ABSTRACT

KEYWORDS:

• Swarm robotics
• deep reinforcement learning
• shepherding
• singling

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Ministerio de Ciencia, Innovación y Universidades (Spain) [project RTI2018-096219-B-I00]. Project co-financed with FEDER funds.

Notes on contributors

Pilar Arques

Pilar Arques received her Ph.D. degree in Computer Engineering And Computing at the University of Alicante (Spain) in 2007. She is now professor at the University of Alicante Polytechnic School in the Department of Computer Science and Artificial Intelligence. Her research interests include swarm robotics, mobile robotics, multi-robot systems, agent systems, reinforcement learning and artificial intelligence.

Fidel Aznar

Fidel Aznar received his Ph.D. degree in Computer Engineering And Computing at the University of Alicante (Spain) in 2006. He is now professor at the University of Alicante Polytechnic School in the Department of Computer Science and Artificial Intelligence. His research interests include swarm robotics, mobile robotics, agent systems, reinforcement learning, deep learning and artificial intelligence.

Mar Pujol

Mar Pujol received her B.A. in Mathematics at the University of Valencia (Spain) in 1985, and the Ph.D. degree in Computer Science at the University of Alicante in 2000. She is full professor at the Department of Computer Science and Artificial Intelligence in the University of Alicante (Spain). Her research interests include swarm robotics, mobile robotics, agent systems, reinforcement learning, deep learning and artificial intelligence.

Ramón Rizo

Ramón Rizo received his Ph.D. in Computer Science from the Polytechnic University of Valencia, Spain, in 1991. He is full professor and doctoral supervisor at the University of Alicante Polytechnic School in the Department of Computer Science and Artificial Intelligence. He has directed 13 doctoral theses, made more than 120 publications and scientific documents in international journals and has participated in more than 130 conferences on topics related to artificial intelligence. His fields of research include deep learning, computer vision, artificial intelligence, and software integration.