21 papers
TwinOR: Photorealistic Digital Twins of Dynamic Operating Rooms for Embodied AI Research
Han Zhang, Yiqing Shen, Roger D. Soberanis-Mukul +11
Developing embodied AI for intelligent surgical systems requires safe, controllable environments for continual learning and evaluation. However, safety regulations and operational…
AffordTissue: Dense Affordance Prediction for Tool-Action Specific Tissue Interaction
Aiza Maksutova, Lalithkumar Seenivasan, Hao Ding +5
Surgical action automation has progressed rapidly toward achieving surgeon-like dexterous control, driven primarily by advances in learning from demonstration and vision-language-a…
Point-Supervised Skeleton-Based Human Action Segmentation
Hongsong Wang, Yiqin Shen, Pengbo Yan +2
Skeleton-based temporal action segmentation is a fundamental yet challenging task, playing a crucial role in enabling intelligent systems to perceive and respond to human activitie…
Counterfactual World Models via Digital Twin-conditioned Video Diffusion
Yiqing Shen, Aiza Maksutova, Chenjia Li +1
World models learn to predict the temporal evolution of visual observations given a control signal, potentially enabling agents to reason about environments through forward simulat…
Text-Driven Reasoning Video Editing via Reinforcement Learning on Digital Twin Representations
Yiqing Shen, Chenjia Li, Mathias Unberath
Text-driven video editing enables users to modify video content only using text queries. While existing methods can modify video content if explicit descriptions of editing targets…
Reasoning Text-to-Video Retrieval via Digital Twin Video Representations and Large Language Models
Yiqing Shen, Chenxiao Fan, Chenjia Li +1
The goal of text-to-video retrieval is to search large databases for relevant videos based on text queries. Existing methods have progressed to handling explicit queries where the…