Robust Multimodal Image Registration Using Deep Recurrent Reinforcement Learning

Abstract

The crucial components of a conventional image registration method are the choice of the right feature representations and similarity measures. These two components, although elaborately designed, are somewhat handcrafted using human knowledge. To this end, these two components are tackled in an end-to-end manner via reinforcement learning in this work. Specifically, an artificial agent, which is composed of a combined policy and value network, is trained to adjust the moving image toward the right direction. We train this network using an asynchronous reinforcement learning algorithm, where a customized reward function is also leveraged to encourage robust image registration. This trained network is further incorporated with a lookahead inference to improve the registration capability. The advantage of this algorithm is fully demonstrated by our superior performance on clinical MR and CT image pairs to other state-of-the-art medical image registration methods.

References in corpus (5)

• A Survey on Deep Learning in Medical Image Analysis
• Asynchronous Methods for Deep Reinforcement Learning
• End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network
• Hierarchical Object Detection with Deep Reinforcement Learning
• Autonomous Driving in Reality with Reinforcement Learning and Image Translation

Cited by in corpus (6)

• Deep Learning in Medical Image Registration: A Review
• Medical Image Registration Using Deep Neural Networks: A Comprehensive Review
• Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey
• Cross-Modality Image Registration using a Training-Time Privileged Third Modality
• Real-time multimodal image registration with partial intraoperative point-set data
• Multi-scale Neural ODEs for 3D Medical Image Registration