GC-MASAC算法下多无人机协同围捕强化学习研究
DOI:
CSTR:
作者:
作者单位:

中国民航大学

作者简介:

通讯作者:

中图分类号:

V249.12;TP181

基金项目:

国家自然科学基金项目(面上项目,重点项目,重大项目)


Reinforcement Learning Research on Multi-UAV Cooperative Encirclement Based on GC-MASAC Algorithm
Author:
Affiliation:

1.Civil Aviation University of China,Tianjin;2.Civil Aviation University of China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对多无人机协同围捕中智能体关系建模不足和奖励信号稀疏的问题,提出一种结合图结构评论家与围捕奖励塑形的多智能体强化学习方法(Graph Critic-Multi- Agent Soft Actor-Critic,GC-MASAC)方法。该方法利用两层图卷积网络聚合多智能体观测—动作信息,并通过基于多追捕者平均目标距离的密集奖励塑形,引导追捕者协同逼近逃逸目标。在多智能体粒子环境(Multi-Agent Particle Environment,MPE)三追一场景中的消融实验表明,训练至1×10?环境交互步时,GC-MASAC的评估回报约为37,较MASAC baseline的?42明显提高;围捕评分达到约0.70,平均距离由1.03降至0.67,团队奖励由?0.98提高至0.76。结果表明,图结构关系建模与奖励塑形能够协同提升二维围捕策略的学习效果和围捕质量。

    Abstract:

    To address inadequate inter-agent relationship modeling and sparse reward signals in multi-UAV cooperative encirclement, a multi-agent reinforcement learning method integrating a graph-structured critic with encir-clement reward shaping, termed Graph Critic–Multi-Agent Soft Actor-Critic (GC-MASAC), is proposed. In the proposed method, multi-agent observation–action information is aggregated using a two-layer graph convolutional network, and coordinated pursuit of the evader is encouraged through dense reward shaping based on the average target distance of multiple pursuers. Ablation experiments are conducted in a three-pursuer–one-evader scenario constructed in the Multi-Agent Particle Environment (MPE). After \(1\times10^6\) environment interaction steps, an evaluation return of approximately 37 is achieved by GC-MASAC, representing a marked improvement over the value of approximately ?42 obtained by the MASAC baseline. An encirclement score of approximately 0.70 is achieved, while the mean distance is reduced from 1.03 to 0.67 and the team reward is increased from ?0.98 to 0.76. The results indicate that the learning performance and encirclement quality of two-dimensional cooperative encirclement policies can be jointly improved through graph-based relational modeling and reward shaping.

    参考文献
    相似文献
    引证文献
引用本文

高志云,唐艺宸,汤新民,等. GC-MASAC算法下多无人机协同围捕强化学习研究[J]. 科学技术与工程, , ():

复制
分享
相关视频

文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2026-06-26
  • 最后修改日期:2026-09-03
  • 录用日期:2026-09-29
  • 在线发布日期:
  • 出版日期:
×
关于抵制伪造参考文献学术不端行为的声明
关于规范使用生成式人工智能的声明
喜报|《科学技术与工程》3篇论文成功入选 “第二十八届中国科协年会影响力提名论文”
2026年会通知 | “技术经济学驱动智能经济生态构建与治理变革”——中国技术经济学会第三十三届学术年会(2026)会议通知暨征文启事(第一轮)
亟待确认版面费归属稿件,敬请作者关注