12 papers
Orca: The World is in Your Mind
Yihao Wang, Yuheng Ji, Mingyu Cao +54
We introduce Orca, an initial instantiation of a general world foundation model. Orca learns a unified world latent space from multimodal world signals and exposes it through multi…
LAST: Bridging Vision-Language and Action Manifolds via Gromov-Wasserstein Alignment
Huaihai Lyu, Chaofan Chen, Yuheng Ji +4
We take a Gromov-Wasserstein perspective on Vision-Language-Action (VLA) learning, where the goal is to make the relational geometry of action representations compatible with the s…
General Covariant Action Modeling: Constructing Generalized Manifolds via Spatio-Temporal Decoupling
Huaihai Lyu, Chaofan Chen, Mingyu Cao +2
Achieving robust generalization from limited data is a central challenge in embodied intelligence. Prevailing methods fail by regressing absolute coordinates, which violates the pr…
PRM-as-a-Judge: A Dense Evaluation Paradigm for Fine-Grained Robotic Auditing
Yuheng Ji, Yuyang Liu, Huajie Tan +15
Current robotic evaluation is still largely dominated by binary success rates, which collapse rich execution processes into a single outcome and obscure critical qualities such as…
RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation
Shihan Wu, Xuecheng Liu, Shaoxuan Xie +81
Despite the critical role of bimanual manipulation in endowing robots with human-like dexterity, large-scale and diverse datasets remain scarce due to the significant hardware hete…
RoboOS-NeXT: A Unified Memory-based Framework for Lifelong, Scalable, and Robust Multi-Robot Collaboration
Huajie Tan, Cheng Chi, Xiansheng Chen +21
The proliferation of collaborative robots across diverse tasks and embodiments presents a central challenge: achieving lifelong adaptability, scalable coordination, and robust sche…
OmniSAT: Compact Action Token, Faster Auto Regression
Huaihai Lyu, Chaofan Chen, Senwei Xie +4
Existing Vision-Language-Action (VLA) models can be broadly categorized into diffusion-based and auto-regressive (AR) approaches: diffusion models capture continuous action distrib…
MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzles
Yuheng Ji, Huajie Tan, Cheng Chi +8
We introduce \textsc{MathSticks}, a benchmark for Visual Symbolic Compositional Reasoning (VSCR), which unifies visual perception, symbolic manipulation, and arithmetic consistency…
EgoPrompt: Prompt Learning for Egocentric Action Recognition
Huaihai Lyu, Chaofan Chen, Yuheng Ji +1
Driven by the increasing demand for applications in augmented and virtual reality, egocentric action recognition has emerged as a prominent research area. It is typically divided i…
VisualTrans: A Benchmark for Real-World Visual Transformation Reasoning
Yuheng Ji, Yipu Wang, Yuyang Liu +5
Visual transformation reasoning (VTR) is a vital cognitive capability that empowers intelligent agents to understand dynamic scenes, model causal relationships, and predict future…
RoboBrain 2.0 Technical Report
BAAI RoboBrain Team, Mingyu Cao, Huajie Tan +50
We introduce RoboBrain 2.0, our latest generation of embodied vision-language foundation models, designed to unify perception, reasoning, and planning for complex embodied tasks in…
RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete
Yuheng Ji, Huajie Tan, Jiayu Shi +14
Recent advancements in Multimodal Large Language Models (MLLMs) have shown remarkable capabilities across various multimodal contexts. However, their application in robotic scenari…