6 papers
M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenes
Zeyu Zhang, Sixu Yan, Muzhi Han +4
We propose M3Bench, a new benchmark for whole-body motion generation in mobile manipulation tasks. Given a 3D scene context, M3Bench requires an embodied agent to reason about its…
Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V
Peiyuan Zhi, Zhiyuan Zhang, Yu Zhao +6
Autonomous robot navigation and manipulation in open environments require reasoning and replanning with closed-loop feedback. In this work, we present COME-robot, the first closed-…
M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenes
Sixu Yan, Zeyu Zhang, Muzhi Han +7
Recent advances in diffusion models have opened new avenues for research into embodied AI agents and robotics. Despite significant achievements in complex robotic locomotion and sk…
LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoning
Shu Wang, Muzhi Han, Ziyuan Jiao +4
Conventional Task and Motion Planning (TAMP) approaches rely on manually crafted interfaces connecting symbolic task planning with continuous motion generation. These domain-specif…
InterPreT: Interactive Predicate Learning from Language Feedback for Generalizable Task Planning
Muzhi Han, Yifeng Zhu, Song-Chun Zhu +2
Learning abstract state representations and knowledge is crucial for long-horizon robot planning. We present InterPreT, an LLM-powered framework for robots to learn symbolic predic…
Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representations
Puhao Li, Tengyu Liu, Yuyang Li +6
Autonomous robotic systems capable of learning novel manipulation tasks are poised to transform industries from manufacturing to service automation. However, modern methods (e.g.,…