11 papers
Harnessing Multimodal Large Language Models for Training-Free Human-Object Interaction Detection
Zhaolin Cai, Huiyu Duan, Liu Yang +5
Human-object interaction (HOI) detection aims to localize human-object pairs and recognize their interactions. Traditional supervised methods perform strongly but rely on task-spec…
Intelligence Across Embodiments
Bo Ai, Henrik I. Christensen, Hao Su
Robotic embodiment encompasses the sensing, kinematics, dynamics, geometry, actuation, and control through which an agent physically interacts with the world. These properties vary…
Rapid Embodiment Adaptation for Quadrupedal Locomotion
Dichen Li, Bo Ai, Nico Bohlinger +3
Humans readily adapt their movements as their bodies change through aging, injury, or load carrying, but learning-based robot policies often break when hardware properties shift. W…
AnyHand: A Large-Scale Synthetic Dataset for RGB(-D) Hand Pose Estimation
Chen Si, Yulin Liu, Bo Ai +4
We present AnyHand, a large-scale synthetic dataset designed to advance the state of the art in 3D hand pose estimation. While recent works with foundation approaches have shown th…
DLEBench: Evaluating Small-scale Object Editing Ability for Instruction-based Image Editing Model
Shibo Hong, Boxian Ai, Jun Kuang +5
Significant progress has been made in the field of Instruction-based Image Editing Models (IIEMs). However, while these models demonstrate plausible adherence to instructions and s…
Scaling Cross-Embodiment World Models for Dexterous Manipulation
Zihao He, Bo Ai, Tongzhou Mu +6
Cross-embodiment learning seeks to build generalist robots that learn from and operate across diverse morphologies, but differences in kinematics and action spaces hinder data shar…
Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations
Shresth Grover, Akshay Gopalkrishnan, Bo Ai +3
Vision-language-action (VLA) models finetuned from vision-language models (VLMs) hold the promise of leveraging rich pretrained representations to build generalist robots across di…
Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation
Qiyue Gao, Xinyu Pi, Kevin Liu +21
Internal world models (WMs) enable agents to understand the world's state and predict transitions, serving as the basis for advanced deliberative reasoning. Recent large Vision-Lan…
Towards Embodiment Scaling Laws in Robot Locomotion
Bo Ai, Liu Dai, Nico Bohlinger +7
Cross-embodiment generalization underpins the vision of building generalist embodied agents for any robot, yet its enabling factors remain poorly understood. We investigate embodim…
Learning Adaptive Dexterous Grasping from Single Demonstrations
Liangzhi Shi, Yulin Liu, Lingqi Zeng +3
How can robots learn dexterous grasping skills efficiently and apply them adaptively based on user instructions? This work tackles two key challenges: efficient skill acquisition f…
Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulation
Tongxuan Tian, Haoyang Li, Bo Ai +3
Cloth manipulation is challenging due to its highly complex dynamics, near-infinite degrees of freedom, and frequent self-occlusions, which complicate both state estimation and dyn…