A Route Planning Method for Intelligent Ships Based on the AS-DDQN Algorithm

Abstract

Addressing the issues in traditional DQN algorithms for intelligent vessel path planning—such as routes overly close to obstacles, frequent sharp turns, slow convergence, overestimation of Q-values, and network instability— this paper proposes an intelligent ship path planning method (AS-DDQN) based on the DDQN algorithm. It combines Angle Searching with DDQN by pre-setting search angles to define search regions, thereby optimizing travel paths, reducing redundant exploration, and accelerating convergence. Additionally, supplementary reward functions are introduced to the traditional reward structure. These include rudder angle deviation penalties, safe distance penalties for the vessel's track, time penalties, and distance-based propulsion rewards. This ensures the vessel maintains a safe distance from obstacles while minimizing steering maneuvers. Finally, simulations were conducted in two distinct maritime environments near Zhoushan with varying complexity levels. Experimental results demonstrate that compared to the DDQN algorithm, the AS-DDQN algorithm significantly accelerates algorithm convergence. Moreover, across different maritime environments, the final planned routes achieved notable effectiveness in ensuring navigation safety and economic efficiency, better meeting the practical navigation requirements of vessels