Deep Homography Estimation in Dynamic Surgical Scenes for Laparoscopic Camera Motion Extraction
arXiv:2109.15098 · doi:10.1080/21681163.2021.2002195
Abstract
Current laparoscopic camera motion automation relies on rule-based approaches or only focuses on surgical tools. Imitation Learning (IL) methods could alleviate these shortcomings, but have so far been applied to oversimplified setups. Instead of extracting actions from oversimplified setups, in this work we introduce a method that allows to extract a laparoscope holder's actions from videos of laparoscopic interventions. We synthetically add camera motion to a newly acquired dataset of camera motion free da Vinci surgery image sequences through a novel homography generation algorithm. The synthetic camera motion serves as a supervisory signal for camera motion estimation that is invariant to object and tool motion. We perform an extensive evaluation of state-of-the-art (SOTA) Deep Neural Networks (DNNs) across multiple compute regimes, finding our method transfers from our camera motion free da Vinci surgery dataset to videos of laparoscopic interventions, outperforming classical homography estimation approaches in both, precision by 41%, and runtime on a CPU by 43%.
Accepted for publication in 2021 AE-CAI Special Issue of the Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization
References in corpus (6)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- 2017 Robotic Instrument Segmentation Challenge
- Image Compositing for Segmentation of Surgical Tools without Manual Annotations
- Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge
- Homography-based Visual Servoing with Remote Center of Motion for Semi-autonomous Robotic Endoscope Manipulation
- ESAD: Endoscopic Surgeon Action Detection Dataset
Cited by in corpus (4)
- Synthetic white balancing for intra-operative hyperspectral imaging
- Homography-based Visual Servoing with Remote Center of Motion for Semi-autonomous Robotic Endoscope Manipulation
- Rapid and robust endoscopic content area estimation: A lean GPU-based pipeline and curated benchmark dataset
- Deep Homography Prediction for Endoscopic Camera Motion Imitation Learning