5 papers
Orca: The World is in Your Mind
Yihao Wang, Yuheng Ji, Mingyu Cao +54
We introduce Orca, an initial instantiation of a general world foundation model. Orca learns a unified world latent space from multimodal world signals and exposes it through multi…
Uni-Skill: Building Self-Evolving Skill Repository for Generalizable Robotic Manipulation
Senwei Xie, Yuntian Zhang, Ruiping Wang +1
While skill-centric approaches leverage foundation models to enhance generalization in compositional tasks, they often rely on fixed skill libraries, limiting adaptability to new t…
OmniSAT: Compact Action Token, Faster Auto Regression
Huaihai Lyu, Chaofan Chen, Senwei Xie +4
Existing Vision-Language-Action (VLA) models can be broadly categorized into diffusion-based and auto-regressive (AR) approaches: diffusion models capture continuous action distrib…
Robotic Programmer: Video Instructed Policy Code Generation for Robotic Manipulation
Senwei Xie, Hongyu Wang, Zhanqi Xiao +2
Zero-shot generalization across various robots, tasks and environments remains a significant challenge in robotic manipulation. Policy code generation methods use executable code t…
M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models
Hongyu Wang, Jiayu Xu, Senwei Xie +6
Multilingual capability is an essential aspect for large multimodal models, since they are usually deployed across various countries and languages. However, most existing benchmark…