78 citations · 140 across the 29 of their papers we have counts for
31 papers
C3VD-DEFCOL: A Deformable Colonoscopy Dataset with Time-Resolved 3D Ground Truth and Realistic Appearance
Ethan Luk, Mayank V. Golhar, Anthony Song +5
3D reconstruction could improve colonoscopy by estimating mucosal coverage and alerting clinicians to missed regions during screening. However, algorithm development is limited as…
Dense Structural Priors for Sparse Functional Landmark Localization in Surgical Videos
Chenyan Jing, Hao Ding, Lalithkumar Seenivasan +2
Vision foundation models such as SAM 3 can provide transferable object-level structure across diverse surgical video conditions, but segmentation outputs do not explicitly encode t…
Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics
Open-H-Embodiment Consortium, :, Nigel Nelson +213
Autonomous medical robots hold promise to improve patient outcomes, reduce provider workload, democratize access to care, and enable superhuman precision. However, autonomous medic…
AffordTissue: Dense Affordance Prediction for Tool-Action Specific Tissue Interaction
Aiza Maksutova, Lalithkumar Seenivasan, Hao Ding +5
Surgical action automation has progressed rapidly toward achieving surgeon-like dexterous control, driven primarily by advances in learning from demonstration and vision-language-a…
Investigating a Policy-Based Formulation for Endoscopic Camera Pose Recovery
Jan Emily Mangulabnan, Akshat Chauhan, Laura Fleig +7
In endoscopic surgery, surgeons continuously locate the endoscopic view relative to the anatomy by interpreting the evolving visual appearance of the intraoperative scene in the co…
SAW: Toward a Surgical Action World Model via Controllable and Scalable Video Generation
Sampath Rapuri, Lalithkumar Seenivasan, Dominik Schneider +9
A surgical world model capable of generating realistic surgical action videos with precise control over tool-tissue interactions can address fundamental challenges in surgical AI a…
Towards Controllable Video Synthesis of Routine and Rare OR Events
Dominik Schneider, Lalithkumar Seenivasan, Sampath Rapuri +8
Purpose: Curating large-scale datasets of operating room (OR) workflow, encompassing rare, safety-critical, or atypical events, remains operationally and ethically challenging. Thi…
DualVision ArthroNav: Investigating Opportunities to Enhance Localization and Reconstruction in Image-based Arthroscopy Navigation via External Cameras
Hongchao Shu, Lalithkumar Seenivasan, Mingxu Liu +6
Arthroscopic procedures can greatly benefit from navigation systems that enhance spatial awareness, depth perception, and field of view. However, existing optical tracking solution…
TwinOR: Photorealistic Digital Twins of Dynamic Operating Rooms for Embodied AI Research
Han Zhang, Yiqing Shen, Roger D. Soberanis-Mukul +11
Developing embodied AI for intelligent surgical systems requires safe, controllable environments for continual learning and evaluation. However, safety regulations and operational…
Investigating Robot Control Policy Learning for Autonomous X-ray-guided Spine Procedures
Florence Klitzner, Blanca Inigo, Benjamin D. Killeen +4
Imitation learning-based robot control policies are enjoying renewed interest in video-based robotics. However, it remains unclear whether this approach applies to X-ray-guided pro…
Constrained Natural Language Action Planning for Resilient Embodied Systems
Grayson Byrd, Corban Rivera, Bethany Kemp +5
Replicating human-level intelligence in the execution of embodied tasks remains challenging due to the unconstrained nature of real-world environments. Novel use of large language…
Egosurg: Arbitrary view synthesis for egocentric replay of operating room workflows from ambient cameras
Han Zhang, Lalithkumar Seenivasan, Jose L. Porras +10
Observing surgical practice has historically relied on fixed vantage points or recollections, leaving the egocentric perspectives that shape clinical decisions undocumented. Ambien…
Explainable AI for Automated User-specific Feedback in Surgical Skill Acquisition
Catalina Gomez, Lalithkumar Seenivasan, Xinrui Zou +9
Traditional surgical skill acquisition relies heavily on expert feedback, yet direct access is limited by faculty availability and variability in subjective assessments. While trai…
Beyond Rigid AI: Towards Natural Human-Machine Symbiosis for Interoperative Surgical Assistance
Lalithkumar Seenivasan, Jiru Xu, Roger D. Soberanis Mukul +6
Emerging surgical data science and robotics solutions, especially those designed to provide assistance in situ, require natural human-machine interfaces to fully unlock their poten…
Position: Foundation Models Need Digital Twin Representations
Yiqing Shen, Hao Ding, Lalithkumar Seenivasan +2
Current foundation models (FMs) rely on token representations that directly fragment continuous real-world multimodal data into discrete tokens. They limit FMs to learning real-wor…
Privacy-Preserving Operating Room Workflow Analysis using Digital Twins
Alejandra Perez, Han Zhang, Yu-Chun Ku +7
The operating room (OR) is a complex environment where optimizing workflows is critical to reduce costs and improve patient outcomes. While computer vision approaches for automatic…
Online Reasoning Video Segmentation with Just-in-Time Digital Twins
Yiqing Shen, Bohan Liu, Chenjia Li +2
Reasoning segmentation (RS) aims to identify and segment objects of interest based on implicit text queries. As such, RS is a catalyst for embodied AI agents, enabling them to inte…
ConceptAgent: LLM-Driven Precondition Grounding and Tree Search for Robust Task Planning and Execution
Corban Rivera, Grayson Byrd, William Paul +12
Robotic planning and execution in open-world environments is a complex problem due to the vast state spaces and high variability of task embodiment. Recent advances in perception a…
StraightTrack: Towards Mixed Reality Navigation System for Percutaneous K-wire Insertion
Han Zhang, Benjamin D. Killeen, Yu-Chun Ku +9
In percutaneous pelvic trauma surgery, accurate placement of Kirschner wires (K-wires) is crucial to ensure effective fracture fixation and avoid complications due to breaching the…
Seamless Augmented Reality Integration in Arthroscopy: A Pipeline for Articular Reconstruction and Guidance
Hongchao Shu, Mingxu Liu, Lalithkumar Seenivasan +5
Arthroscopy is a minimally invasive surgical procedure used to diagnose and treat joint problems. The clinical workflow of arthroscopy typically involves inserting an arthroscope i…
Towards Robust Surgical Automation via Digital Twin Representations from Foundation Models
Hao Ding, Lalithkumar Seenivasan, Hongchao Shu +7
Large language model-based (LLM) agents are emerging as a powerful enabler of robust embodied intelligence due to their capability of planning complex action sequences. Sound plann…
Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgery
Long Bai, Guankun Wang, Mobarakol Islam +3
Medical visual question answering (VQA) bridges the gap between visual information and clinical decision-making, enabling doctors to extract understanding from clinical images and…
SegSTRONG-C: Segmenting Surgical Tools Robustly On Non-adversarial Generated Corruptions -- An EndoVis'24 Challenge
Hao Ding, Yuqian Zhang, Tuxun Lu +39
Surgical data science has seen rapid advancement with the excellent performance of end-to-end deep neural networks (DNNs). Despite their successes, DNNs have been proven susceptibl…
Surgical-VQLA: Transformer with Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgery
Long Bai, Mobarakol Islam, Lalithkumar Seenivasan +1
Despite the availability of computer-aided simulators and recorded videos of surgical procedures, junior residents still heavily rely on experts to answer their queries. However, e…