
科学技术与工程

中国技术经济学会
面向经腔道自主规划的残差强化学习方法
Residual Reinforcement Learning for Autonomous Transluminal Intervention
使用连续体机器人的经自然腔道介入面临介入路径曲折狭窄和腔道软组织挤压受力等挑战。针对介入递送过程中,现有规划方法难以兼顾多个控制目标,导致难以到达较深位置的问题,提出一种基于残差强化学习的自主规划方案。该方法能够实现柔性连续体机器人经自然腔道的自主递送。通过建立连续体机器人递送姿态与自然腔道空间状态间的反馈偏差模型来控制递送过程中的姿态目标。同时建立连续体机器人的整体运动过程的马尔可夫模型,用于强化学习算法的训练过程。利用姿态反馈控制与强化学习控制相结合产生的残差策略来输出连续体机器人递送过程的最优动作。在仿真支气管腔道中的实验表明,所提出的方法比现有方法的收敛速度快60%以上,能够以平滑、无碰撞的轨迹规划连续体机器人的经腔道介入过程,在多个指标方面优于现有方法。
The natural orifice intervention using continuum robots faces challenges such as tortuous and narrow intervention paths, as well as compressive forces exerted by soft tissues in the orifice. To address the issue in the delivery process where existing planning methods struggle to balance multiple control objectives, resulting in difficulty in reaching deeper positions, an autonomous planning scheme based on residual reinforcement learning was proposed. The method enables the autonomous delivery of continuum robots through natural orifices. A feedback deviation model between the delivery posture of the continuum robot and the spatial state of the natural orifice was established to control the posture target during the delivery process. Simultaneously, a Markov model of the overall motion process of the continuum robot was constructed to train the reinforcement learning algorithm. A residual strategy, generated by combining posture feedback control with reinforcement learning control, was used to output the optimal actions for the continuum robot's delivery process. Experiments conducted in a simulated bronchial orifice show that the proposed method converges over 60% faster than existing methods and can plan smooth, collision-free trajectories for the continuum robot's intervention through the orifice, outperforming existing methods in several key metrics.
连续体机器人 / 自主规划 / 残差策略 / 经腔道介入 / 强化学习
continuum robot / autonomous planning / residual strategy / transluminal intervention / reinforcement learning
/
| 〈 |
|
〉 |