Deep Homography Prediction for Endoscopic Camera Motion Imitation Learning

Abstract

In this work, we investigate laparoscopic camera motion automation through imitation learning from retrospective videos of laparoscopic interventions. A novel method is introduced that learns to augment a surgeon's behavior in image space through object motion invariant image registration via homographies. Contrary to existing approaches, no geometric assumptions are made and no depth information is necessary, enabling immediate translation to a robotic setup. Deviating from the dominant approach in the literature which consist of following a surgical tool, we do not handcraft the objective and no priors are imposed on the surgical scene, allowing the method to discover unbiased policies. In this new research field, significant improvements are demonstrated over two baselines on the Cholec80 and HeiChole datasets, showcasing an improvement of 47% over camera motion continuation. The method is further shown to indeed predict camera motion correctly on the public motion classification labels of the AutoLaparo dataset. All code is made accessible on GitHub.

arXiv commentEarly accepted at MICCAI 2023

References in corpus (4)

• Robotic Endoscope Control via Autonomous Instrument Tracking
• Deep Homography Estimation in Dynamic Surgical Scenes for Laparoscopic Camera Motion Extraction
• Homography-based Visual Servoing with Remote Center of Motion for Semi-autonomous Robotic Endoscope Manipulation
• Rapid and robust endoscopic content area estimation: A lean GPU-based pipeline and curated benchmark dataset