Deep reinforcement learning for guidewire navigation in coronary artery phantom

Abstract

In percutaneous intervention for treatment of coronary plaques, guidewire navigation is a primary procedure for stent delivery. Steering a flexible guidewire within coronary arteries requires considerable training, and the non-linearity between the control operation and the movement of the guidewire makes precise manipulation difficult. Here, we introduce a deep reinforcement learning(RL) framework for autonomous guidewire navigation in a robot-assisted coronary intervention. Using Rainbow, a segment-wise learning approach is applied to determine how best to accelerate training using human demonstrations with deep Q-learning from demonstrations (DQfD), transfer learning, and weight initialization. `State' for RL is customized as a focus window near the guidewire tip, and subgoals are placed to mitigate a sparse reward problem. The RL agent improves performance, eventually enabling the guidewire to reach all valid targets in `stable' phase. Our framework opens anew direction in the automation of robot-assisted intervention, providing guidance on RL in physical spaces involving mechanical fatigue.

arXiv comment15 pages, 7 figures, 3 tables

References in corpus (10)

• Prioritized Experience Replay
• Dueling Network Architectures for Deep Reinforcement Learning
• End-to-End Training of Deep Visuomotor Policies
• Noisy Networks for Exploration
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Deep Q-learning from Demonstrations
• Challenges of Real-World Reinforcement Learning
• Reinforcement Learning from Imperfect Demonstrations
• Robust Imitation of Diverse Behaviors
• Dynamic Coronary Roadmapping via Catheter Tip Tracking in X-ray Fluoroscopy with Deep Learning Based Bayesian Filtering

Cited by in corpus (5)

• Autonomous Navigation for Robot-assisted Intraluminal and Endovascular Procedures: A Systematic Review
• Artificial Intelligence in the Autonomous Navigation of Endovascular Interventions: A Systematic Review
• Autonomous navigation of catheters and guidewires in mechanical thrombectomy using inverse reinforcement learning
• Reinforcement Learning for Safe Autonomous Two Device Navigation of Cerebral Vessels in Mechanical Thrombectomy
• Robust Path Planning via Learning from Demonstrations for Robotic Catheters in Deformable Environments