Potential field hierarchical reinforcement learning approach for target search by multi-AUV in 3-D underwater environments

ABSTRACT

Multiple autonomous underwater vehicles (multi-AUV) target search is the important element to realise underwater rescue, underwater detection. To improve the efficiency of the multi-AUV target search in three-dimensional underwater environments, a potential field hierarchical reinforcement learning approach is proposed in this paper. Unlike other algorithms that need repeated training in the choice of parameters, the proposed approach obtains all the required parameters automatically through learning. By integrating segmental options with the traditional hierarchy reinforcement learning (HRL) algorithm, the potential field hierarchy is built. The potential field is implemented in the parameters of the HRL, which provides with reasonable paths of the target search for the unexplored environments. In search tasks, the designed method can control the multi-AUV system to find the target effectively. The simulation results show that the proposed approach is capable of controlling multi-AUV to achieve search task of multiple targets with higher efficiency and adaptability compared with the HRL algorithm and the lawn-mowing algorithm.

KEYWORDS:

• Target search
• multi-AUV
• hierarchy reinforcement learning
• potential field
• 3-D underwater environments

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by China Postdoctoral Science Foundation [grant number 2017M621587]; National Natural Science Foundation of China [grant number 61773177]; Natural Science Foundation of Jiangsu Province [grant number BK20171270, BK20141253]; Jiangsu Planned Projects for Postdoctoral Research Funds [grant number 1701076B]; Key University Science Research Project of Jiangsu Province [grant number 15KJA460004].