2018 Robotic Scene Segmentation Challenge

Abstract

In 2015 we began a sub-challenge at the EndoVis workshop at MICCAI in Munich using endoscope images of ex-vivo tissue with automatically generated annotations from robot forward kinematics and instrument CAD models. However, the limited background variation and simple motion rendered the dataset uninformative in learning about which techniques would be suitable for segmentation in real surgery. In 2017, at the same workshop in Quebec we introduced the robotic instrument segmentation dataset with 10 teams participating in the challenge to perform binary, articulating parts and type segmentation of da Vinci instruments. This challenge included realistic instrument motion and more complex porcine tissue as background and was widely addressed with modifications on U-Nets and other popular CNN architectures. In 2018 we added to the complexity by introducing a set of anatomical objects and medical devices to the segmented classes. To avoid over-complicating the challenge, we continued with porcine data which is dramatically simpler than human tissue due to the lack of fatty tissue occluding many organs.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Deep Residual Learning for Image Recognition
• Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning
• Xception: Deep Learning with Depthwise Separable Convolutions
• Pyramid Scene Parsing Network
• 2017 Robotic Instrument Segmentation Challenge

Cited by in corpus (25)

• Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review
• Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videos
• Deep learning for detection and segmentation of artefact and disease instances in gastrointestinal endoscopy
• Recent Progress in Transformer-based Medical Image Analysis
• Towards Holistic Surgical Scene Understanding
• Deep Learning for Surgical Instrument Recognition and Segmentation in Robotic-Assisted Surgeries: A Systematic Review
• Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the art
• CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection
• Global-Reasoned Multi-Task Learning Model for Surgical Scene Understanding
• MATIS: Masked-Attention Transformers for Surgical Instrument Segmentation
• Multi-frame Feature Aggregation for Real-time Instrument Segmentation in Endoscopic Video
• Dynamic Interactive Relation Capturing via Scene Graph Learning for Robotic Surgical Report Generation
• Deep Homography Estimation in Dynamic Surgical Scenes for Laparoscopic Camera Motion Extraction
• Multitask Learning in Minimally Invasive Surgical Vision: A Review
• SurgRIPE challenge: Benchmark of Surgical Robot Instrument Pose Estimation
• Zero-shot 3D Segmentation of Abdominal Organs in CT Scans Using Segment Anything Model 2: Adapting Video Tracking Capabilities for 3D Medical Imaging
• One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video
• LC-GAN: Image-to-image Translation Based on Generative Adversarial Network for Endoscopic Images
• Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusion
• How can we learn (more) from challenges? A statistical approach to driving future algorithm development
• Learning Domain Adaptation with Model Calibration for Surgical Report Generation in Robotic Surgery
• 2020 CATARACTS Semantic Segmentation Challenge
• Class-Incremental Domain Adaptation with Smoothing and Calibration for Surgical Report Generation
• Noisy-LSTM: Improving Temporal Awareness for Video Semantic Segmentation
• Learning and Reasoning with the Graph Structure Representation in Robotic Surgery