5 papers
PhysCodeGym: Benchmarking Coding Agents for Physically Verifiable 4D Reconstruction
Shuai Shao, Jinchen He, Cunxin Fan +10
Building physical simulations from video requires coding agents to translate observations into executable models and refine them through tool use. We introduce PhysCodeGym, a bench…
Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization
Jiaming Zhou, Qihang Zhang, Gangwei Xu +9
Zero-shot cross-task generalization, where a policy must execute manipulation tasks never seen during training, remains a central challenge in robot learning. In large language mod…
GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization
Xiaosong Jia, Bowen Yang, Zuhao Ge +17
Vision-Language-Action (VLA) models aim for general robot learning by aligning action as a modality within powerful Vision-Language Models (VLMs). Existing VLAs rely on end-to-end…
Interleave-VLA: Enhancing Robot Manipulation with Interleaved Image-Text Instructions
Cunxin Fan, Xiaosong Jia, Yihang Sun +8
The rise of foundation models paves the way for generalist robot policies in the physical world. Existing methods relying on text-only instructions often struggle to generalize to…
WOMD-Reasoning: A Large-Scale Dataset for Interaction Reasoning in Driving
Yiheng Li, Cunxin Fan, Chongjian Ge +9
Language models uncover unprecedented abilities in analyzing driving scenarios, owing to their limitless knowledge accumulated from text-based pre-training. Naturally, they should…