6 papers
SCOPE: Scene-graph Construction Optimized by Plan Error-correction for Robot Long-term Planning Datasets
Chaoyue Li, Zeyu Gao, Yanfeng Lu +1
Task and Motion Planning (TAMP) stands as a fundamental challenge in robotics, essential for enabling autonomous agents to execute complex, long-horizon tasks in diverse environmen…
SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning
Chaoyue Li, Shoudong Han, Haoyang Wu +4
Large multimodal models (LMMs) need to understand how entities move and interact as a scene develops. This requires following the same entities over time and relating their observe…
GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulation
Yupeng Zheng, Xiang Li, Songen Gu +11
Vision-language pre-training and predictive world modeling provide robot policies with rich semantic and dynamic visual features, but their native action and visual-prediction obje…
WALA Learning Executable Latent Actions from Action-Labeled Demonstrations and Action-Free Videos
Jiahao Liu, Zhongpu Xia, Yupeng Zheng +14
Human videos provide rich information about object manipulation at a scale difficult to reproduce with robots, but often lack action annotations that can directly supervise robot p…
4DVLT: Dynamic Scene Understanding with Worldline-Centered Vision-Language Tracking
Chaoyue Li, Boxue Yang, Shengyao Zhou +4
Language-guided dynamic scene understanding requires identifying one physical entity across time, metric 3D space, and multiple viewpoints, a setting not captured by current multim…
LMM-Track4D: Eliciting 4D Dynamic Reasoning in LMMs via Trajectory-Grounded Dialogue
Chaoyue Li, Yongxue Xu, Jie Feng +1
Recent large multimodal models (LMMs) have become increasingly capable on image and video understanding, yet still struggle to sustain 4D continuous spatiotemporal dynamic reasonin…