CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions

Abstract

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice. While similar benefits are longed for in the field of interventional imaging, this ambition is challenged by a much higher heterogeneity. Clinical workflows within interventional suites and operating theatres are extremely complex and typically rely on poorly integrated intra-operative devices, sensors, and support infrastructures. Taking stock of some of the most exciting developments in machine learning and artificial intelligence for computer assisted interventions, we highlight the crucial need to take context and human factors into account in order to address these challenges. Contextual artificial intelligence for computer assisted intervention, or CAI4CAI, arises as an emerging opportunity feeding into the broader field of surgical data science. Central challenges being addressed in CAI4CAI include how to integrate the ensemble of prior knowledge and instantaneous sensory information from experts, sensors and actuators; how to create and communicate a faithful and actionable shared representation of the surgery among a mixed human-AI actor team; how to design interventional systems and associated cognitive shared control schemes for online uncertainty-aware collaborative decision making ultimately producing more precise and reliable interventions.

References in corpus (4)

• On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task
• Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detection
• Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy
• Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks

Cited by in corpus (13)

• Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videos
• CholecTriplet2021: A benchmark challenge for surgical action triplet recognition
• LoViT: Long Video Transformer for Surgical Phase Recognition
• CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection
• Rendezvous in Time: An Attention-based Temporal Fusion approach for Surgical Triplet Recognition
• Encoding Surgical Videos as Latent Spatiotemporal Graphs for Object and Anatomy-Driven Reasoning
• Weakly Supervised Temporal Convolutional Networks for Fine-grained Surgical Activity Recognition
• CholecInstanceSeg: A Tool Instance Segmentation Dataset for Laparoscopic Surgery
• Multi-Task Temporal Convolutional Networks for Joint Recognition of Surgical Phases and Steps in Gastric Bypass Procedures
• Multitask Learning in Minimally Invasive Surgical Vision: A Review
• Surgical Text-to-Image Generation
• Early Operative Difficulty Assessment in Laparoscopic Cholecystectomy via Snapshot-Centric Video Analysis
• Does anatomical contextual information improve 3D U-Net based brain tumor segmentation?