42 citations · 57 across the 16 of their papers we have counts for
15 papers · 1 filter
MintAct: A Unified Visual Agent for Digital Environments
Mingfei Gao, Rui Tian, Haiming Gang +11
We present MintAct, a family of vision-language models that unifies UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use, trained at 2B, 4B, and…
OR-Action: Multi-Role Video Understanding with Fine-Grained Actions
Felix Tristram, Ege Özsoy, Christian Benz +3
Fine-grained understanding of operating room (OR) activity could enable workflow-aware assistance, yet remains difficult due to clutter, occlusions, and limited sensing. The prevai…
PanORama: Multiview Consistent Panoptic Segmentation in Operating Rooms
Tuna Gürbüz, Ege Özsoy, Tony Danjun Wang +1
Operating rooms (ORs) are cluttered, dynamic, highly occluded environments, where reliable spatial understanding is essential for situational awareness during complex surgical work…
Recognizing Surgical Phases Anywhere: Few-Shot Test-time Adaptation and Task-graph Guided Refinement
Kun Yuan, Tingxuan Chen, Shi Li +9
The complexity and diversity of surgical workflows, driven by heterogeneous operating room settings, institutional protocols, and anatomical variability, present a significant chal…
EgoExOR: An Ego-Exo-Centric Operating Room Dataset for Surgical Activity Understanding
Ege Özsoy, Arda Mamur, Felix Tristram +4
Operating rooms (ORs) demand precise coordination among surgeons, nurses, and equipment in a fast-paced, occlusion-heavy environment, necessitating advanced perception models to en…
Specialized Foundation Models for Intelligent Operating Rooms
Ege Özsoy, Chantal Pellegrini, David Bani-Harouni +3
Surgical procedures unfold in complex environments demanding coordination between surgical teams, tools, imaging and increasingly, intelligent robotic systems. Ensuring safety and…
MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environments
Ege Özsoy, Chantal Pellegrini, Tobias Czempiel +7
Operating rooms (ORs) are complex, high-stakes environments requiring precise understanding of interactions among medical staff, tools, and equipment for enhancing surgical assista…
ORacle: Large Vision-Language Models for Knowledge-Guided Holistic OR Domain Modeling
Ege Özsoy, Chantal Pellegrini, Matthias Keicher +1
Every day, countless surgeries are performed worldwide, each within the distinct settings of operating rooms (ORs) that vary not only in their setups but also in the personnel, too…
Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting
Chantal Pellegrini, Matthias Keicher, Ege Özsoy +1
Radiology reporting is a crucial part of the communication between radiologists and other medical professionals, but it can be time-consuming and error-prone. One approach to allev…
Whether and When does Endoscopy Domain Pretraining Make Sense?
Dominik Batić, Felix Holm, Ege Özsoy +2
Automated endoscopy video analysis is a challenging task in medical computer vision, with the primary objective of assisting surgeons during procedures. The difficulty arises from…
LABRAD-OR: Lightweight Memory Scene Graphs for Accurate Bimodal Reasoning in Dynamic Operating Rooms
Ege Özsoy, Tobias Czempiel, Felix Holm +2
Modern surgeries are performed in complex and dynamic settings, including ever-changing interactions between medical staff, patients, and equipment. The holistic modeling of the op…
Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosis
Chantal Pellegrini, Matthias Keicher, Ege Özsoy +3
Automated diagnosis prediction from medical images is a valuable resource to support clinical decision-making. However, such systems usually need to be trained on large amounts of…
Location-Free Scene Graph Generation
Ege Özsoy, Felix Holm, Mahdi Saleh +4
Scene Graph Generation (SGG) is a visual understanding task, aiming to describe a scene as a graph of entities and their relationships with each other. Existing works rely on locat…
4D-OR: Semantic Scene Graphs for OR Domain Modeling
Ege Özsoy, Evin Pınar Örnek, Ulrich Eck +3
Surgical procedures are conducted in highly complex operating rooms (OR), comprising different actors, devices, and interactions. To date, only medically trained human experts are…
Multimodal Semantic Scene Graphs for Holistic Modeling of Surgical Procedures
Ege Özsoy, Evin Pınar Örnek, Ulrich Eck +2
From a computer science viewpoint, a surgical domain model needs to be a conceptual one incorporating both behavior and data. It should therefore model actors, devices, tools, thei…