Abstract: The ATMADDPG ( Attention Mechanism based Multi-Agent Deep Deterministic Policy Gradient) algorithm is proposed to improve the navigation ability of a multi-unmanned ship formation system. In the training phase, the algorithm trains the best strategy through a large number of experiments, and directly uses the trained best strategy to obtain the best formation path in the experimental phase. The simulation experiment uses four ' Baichuan' unmanned ships as experimental objects. The experimental results show that the formation maintenance strategy based on the ATMADDPG algorithm can achieve stable navigation of multiple unmanned ship formations and meet the requirements of formation maintenance to some extent. Compared to the MADDPG (Multi-Agent Depth Deterministic Policy Gradient ) algorithm, the developed ATMADDPG algorithm shows superior performance in terms of convergence speed, formation maintenance ability, and adaptability to environmental changes. The comprehensive navigation efficiency can be improved by about 80% , which has great application potential.

Key words: formation navigation of multi-unmanned surface vehicles, multi-agent depth deterministic policy gradient (MADDPG) algorithm, attention mechanism, deep reinforcement learning