Bimodal Camera Pose Prediction for Endoscopy

Abstract

Deducing the 3D structure of endoscopic scenes from images is exceedingly challenging. In addition to deformation and view-dependent lighting, tubular structures like the colon present problems stemming from their self-occluding and repetitive anatomical structure. In this paper, we propose SimCol, a synthetic dataset for camera pose estimation in colonoscopy, and a novel method that explicitly learns a bimodal distribution to predict the endoscope pose. Our dataset replicates real colonoscope motion and highlights the drawbacks of existing methods. We publish 18k RGB images from simulated colonoscopy with corresponding depth and camera poses and make our data generation environment in Unity publicly available. We evaluate different camera pose prediction methods and demonstrate that, when trained on our data, they generalize to real colonoscopy sequences, and our bimodal approach outperforms prior unimodal work.

This article has been accepted for publication in IEEE Transactions on Medical Robotics and Bionics. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/TMRB.2023.3320267

References in corpus (4)

• Deep EndoVO: A Recurrent Convolutional Neural Network (RCNN) based Visual Odometry Approach for Endoscopic Capsule Robots
• EndoMapper dataset of complete calibrated endoscopy procedures
• SLAM based Quasi Dense Reconstruction For Minimally Invasive Surgery Scenes
• SuperPoint features in endoscopy

Cited by in corpus (5)

• Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registration
• Tracking and Mapping in Medical Computer Vision: A Review
• SimCol3D -- 3D Reconstruction during Colonoscopy Challenge
• LightDepth: Single-View Depth Self-Supervision from Illumination Decline
• Task-guided Domain Gap Reduction for Monocular Depth Prediction in Endoscopy