578 citations · 735 across the 25 of their papers we have counts for
27 papers
SurgNarrator: A Generative Retrieval Framework for Surgical Video Understanding
Yuqing Feng, Jiawei Ma, Kevin Qinghong Lin +6
Surgical procedures unfold as structured and recurring clinical events, whose real-time understanding via intraoperative surgical videos is critical for intraoperative decision-mak…
HyperVLP: Enhancing Hierarchical Surgical Video-Language Pre-training in Hyperbolic Space
Yaojun Hu, Kun Yuan, Nassir Navab +3
Surgical vision-language foundation models typically adopt educational materials, such as surgical lecture videos, to transfer surgical knowledge encoded in language into visual re…
CliPPER: Contextual Video-Language Pretraining on Long-form Intraoperative Surgical Procedures for Event Recognition
Florian Stilz, Vinkle Srivastav, Nassir Navab +1
Video-language foundation models have proven to be highly effective in zero-shot applications across a wide range of tasks. A particularly challenging area is the intraoperative su…
Where It Moves, It Matters: Referring Surgical Instrument Segmentation via Motion
Meng Wei, Kun Yuan, Shi Li +7
Enabling intuitive, language-driven interaction with surgical scenes is a critical step toward intelligent operating rooms and autonomous surgical robotic assistance. However, the…
From Panel to Pixel: Zoom-In Vision-Language Pretraining from Biomedical Scientific Literature
Kun Yuan, Min Woo Sun, Zhen Chen +7
There is a growing interest in developing strong biomedical vision-language models. A popular approach to achieve robust representations is to use web-scale scientific data. Howeve…
DSeq-JEPA: Discriminative Sequential Joint-Embedding Predictive Architecture
Xiangteng He, Shunsuke Sakai, Shivam Chandhok +5
Recent advances in self-supervised visual representation learning have demonstrated the effectiveness of predictive latent-space objectives for learning transferable features. In p…
End-to-End Learning of Multi-Organ Implicit Surfaces from 3D Medical Imaging Data
Farahdiba Zarin, Nicolas Padoy, Jérémy Dana +1
The fine-grained surface reconstruction of different organs from 3D medical imaging can provide advanced diagnostic support and improved surgical planning. However, the representat…
Recognizing Surgical Phases Anywhere: Few-Shot Test-time Adaptation and Task-graph Guided Refinement
Kun Yuan, Tingxuan Chen, Shi Li +9
The complexity and diversity of surgical workflows, driven by heterogeneous operating room settings, institutional protocols, and anatomical variability, present a significant chal…
Adaptation of Multi-modal Representation Models for Multi-task Surgical Computer Vision
Soham Walimbe, Britty Baby, Vinkle Srivastav +1
Surgical AI often involves multiple tasks within a single procedure, like phase recognition or assessing the Critical View of Safety in laparoscopic cholecystectomy. Traditional mo…
Multi-modal Representations for Fine-grained Multi-label Critical View of Safety Recognition
Britty Baby, Vinkle Srivastav, Pooja P. Jain +3
The Critical View of Safety (CVS) is crucial for safe laparoscopic cholecystectomy, yet assessing CVS criteria remains a complex and challenging task, even for experts. Traditional…
Learning from Sparse Point Labels for Dense Carcinosis Localization in Advanced Ovarian Cancer Assessment
Farahdiba Zarin, Riccardo Oliva, Vinkle Srivastav +7
Learning from sparse labels is a challenge commonplace in the medical domain. This is due to numerous factors, such as annotation cost, and is especially true for newly introduced…
SurgTPGS: Semantic 3D Surgical Scene Understanding with Text Promptable Gaussian Splatting
Yiming Huang, Long Bai, Beilei Cui +6
In contemporary surgical research and practice, accurately comprehending 3D surgical scenes with text-promptable capabilities is particularly crucial for surgical planning and real…
SurgVidLM: Towards Multi-grained Surgical Video Understanding with Large Language Model
Guankun Wang, Junyi Wang, Wenjin Mo +11
Surgical scene understanding is critical for surgical training and robotic decision-making in robot-assisted surgery. Recent advances in Multimodal Large Language Models (MLLMs) ha…
Text-driven Adaptation of Foundation Models for Few-shot Surgical Workflow Analysis
Tingxuan Chen, Kun Yuan, Vinkle Srivastav +2
Purpose: Surgical workflow analysis is crucial for improving surgical efficiency and safety. However, previous studies rely heavily on large-scale annotated datasets, posing challe…
Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment
Yaling Shen, Zhixiong Zhuang, Kun Yuan +4
Medical multimodal large language models (MLLMs) are becoming an instrumental part of healthcare systems, assisting medical personnel with decision making and results analysis. Mod…
EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgery
Guankun Wang, Long Bai, Junyi Wang +13
Recently, Multimodal Large Language Models (MLLMs) have demonstrated their immense potential in computer-aided diagnosis and decision-making. In the context of robotic-assisted sur…
OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining
Ming Hu, Kun Yuan, Yaling Shen +17
Surgical practice involves complex visual interpretation, procedural skills, and advanced medical knowledge, making surgical vision-language pretraining (VLP) particularly challeng…
Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation
Kun Yuan, Vinkle Srivastav, Nassir Navab +1
Surgical video-language pretraining (VLP) faces unique challenges due to the knowledge domain gap and the scarcity of multi-modal data. This study aims to bridge the gap by address…
HecVL: Hierarchical Video-Language Pretraining for Zero-shot Surgical Phase Recognition
Kun Yuan, Vinkle Srivastav, Nassir Navab +1
Natural language could play an important role in developing generalist surgical models by providing a broad source of supervision from raw texts. This flexible form of supervision…
Enhancing Gait Video Analysis in Neurodegenerative Diseases by Knowledge Augmentation in Vision Language Model
Diwei Wang, Kun Yuan, Candice Muller +3
We present a knowledge augmentation strategy for assessing the diagnostic groups and gait impairment from monocular gait videos. Based on a large-scale pre-trained Vision Language…
Advancing Surgical VQA with Scene Graph Knowledge
Kun Yuan, Manasi Kattel, Joel L. Lavanchy +3
Modern operating room is becoming increasingly complex, requiring innovative intra-operative support systems. While the focus of surgical data science has largely been on video ana…
Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures
Kun Yuan, Vinkle Srivastav, Tong Yu +5
Recent advancements in surgical computer vision applications have been driven by vision-only models, which do not explicitly integrate the rich semantics of language into their des…
Intuitive Surgical SurgToolLoc and SurgVU Challenges Results: 2022-2025
Aneeq Zia, Max Berniker, Rogerio Garcia Nespolo +153
Robotic assisted (RA) surgery promises to transform surgical intervention. Intuitive Surgical is committed to fostering these changes and the machine learning models and algorithms…
Surgical Data Science -- from Concepts toward Clinical Translation
Lena Maier-Hein, Matthias Eisenmann, Duygu Sarikaya +47
Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a…