面向经腔道自主规划的残差强化学习方法

汤向荣 ,  边桂彬 ,  李桢 ,  马睿宸

科学技术与工程 ›› 2025, Vol. 25 ›› Issue (17) :7244-7251

PDF (5575KB)
科学技术与工程 ›› 2025, Vol. 25 ›› Issue (17) :7244-7251 DOI: 10.12404/j.issn.1671-1815.2404338
自动化技术、计算机技术

面向经腔道自主规划的残差强化学习方法

作者信息 +

Residual Reinforcement Learning for Autonomous Transluminal Intervention

Author information +
文章历史 +

摘要

使用连续体机器人的经自然腔道介入面临介入路径曲折狭窄和腔道软组织挤压受力等挑战。针对介入递送过程中,现有规划方法难以兼顾多个控制目标,导致难以到达较深位置的问题,提出一种基于残差强化学习的自主规划方案。该方法能够实现柔性连续体机器人经自然腔道的自主递送。通过建立连续体机器人递送姿态与自然腔道空间状态间的反馈偏差模型来控制递送过程中的姿态目标。同时建立连续体机器人的整体运动过程的马尔可夫模型,用于强化学习算法的训练过程。利用姿态反馈控制与强化学习控制相结合产生的残差策略来输出连续体机器人递送过程的最优动作。在仿真支气管腔道中的实验表明,所提出的方法比现有方法的收敛速度快60%以上,能够以平滑、无碰撞的轨迹规划连续体机器人的经腔道介入过程,在多个指标方面优于现有方法。

Abstract

The natural orifice intervention using continuum robots faces challenges such as tortuous and narrow intervention paths, as well as compressive forces exerted by soft tissues in the orifice. To address the issue in the delivery process where existing planning methods struggle to balance multiple control objectives, resulting in difficulty in reaching deeper positions, an autonomous planning scheme based on residual reinforcement learning was proposed. The method enables the autonomous delivery of continuum robots through natural orifices. A feedback deviation model between the delivery posture of the continuum robot and the spatial state of the natural orifice was established to control the posture target during the delivery process. Simultaneously, a Markov model of the overall motion process of the continuum robot was constructed to train the reinforcement learning algorithm. A residual strategy, generated by combining posture feedback control with reinforcement learning control, was used to output the optimal actions for the continuum robot's delivery process. Experiments conducted in a simulated bronchial orifice show that the proposed method converges over 60% faster than existing methods and can plan smooth, collision-free trajectories for the continuum robot's intervention through the orifice, outperforming existing methods in several key metrics.

关键词

连续体机器人 / 自主规划 / 残差策略 / 经腔道介入 / 强化学习

Key words

continuum robot / autonomous planning / residual strategy / transluminal intervention / reinforcement learning

引用本文

引用格式 ▾
汤向荣, 边桂彬, 李桢, 马睿宸. 面向经腔道自主规划的残差强化学习方法[J]. 科学技术与工程, 2025, 25(17): 7244-7251 DOI:10.12404/j.issn.1671-1815.2404338
TANG Xiang-rong, BIAN Gui-bin, LI Zhen, MA Rui-chen. Residual Reinforcement Learning for Autonomous Transluminal Intervention[J]. Science Technology and Engineering, 2025, 25(17): 7244-7251 DOI:10.12404/j.issn.1671-1815.2404338

登录浏览全文

4963

注册一个新账户 忘记密码

参考文献

基金资助

北京市杰出青年科学基金(JQ21016)

PDF (5575KB)

38

访问

0

被引

详细

导航
相关文章

/

〈 〉