Abstract:To address inadequate inter-agent relationship modeling and sparse reward signals in multi-UAV cooperative encirclement, a multi-agent reinforcement learning method integrating a graph-structured critic with encir-clement reward shaping, termed Graph Critic–Multi-Agent Soft Actor-Critic (GC-MASAC), is proposed. In the proposed method, multi-agent observation–action information is aggregated using a two-layer graph convolutional network, and coordinated pursuit of the evader is encouraged through dense reward shaping based on the average target distance of multiple pursuers. Ablation experiments are conducted in a three-pursuer–one-evader scenario constructed in the Multi-Agent Particle Environment (MPE). After \(1\times10^6\) environment interaction steps, an evaluation return of approximately 37 is achieved by GC-MASAC, representing a marked improvement over the value of approximately ?42 obtained by the MASAC baseline. An encirclement score of approximately 0.70 is achieved, while the mean distance is reduced from 1.03 to 0.67 and the team reward is increased from ?0.98 to 0.76. The results indicate that the learning performance and encirclement quality of two-dimensional cooperative encirclement policies can be jointly improved through graph-based relational modeling and reward shaping.