18 papers
PerturBot: Breaking Shortcut Priors in Vision-Language-Action Models with Perturbative Training
Mingyu Liu, Chonghao Sima, Tianjian Feng +4
A vision--language--action (VLA) policy can complete complex tasks while ignoring the evidence that should determine its actions. An object held near the wrist camera can displace…
Open-AoE: An Open Egocentric Manipulation Dataset and Toolchain for Embodied Learning
Zishuo Li, Bowen Yang, Changtao Miao +29
Egocentric videos of human manipulation provide scalable supervision for embodied intelligence, yet existing resources rarely combine low-cost continuous capture, manipulation-leve…
CoRE-VLA: Towards Scalable and Robust Vision-Language-Action Modeling via Conditional Routing of Experts
Haozhe Zhang, Sixian Li, Yifei Zhang +5
Vision-language-action (VLA) models have advanced generalist robotic manipulation, yet real-world deployment reveals a fundamental challenge: robots are equipped with diverse and h…
RegionReasoner: Region-Grounded Multi-Round Visual Reasoning
Wenfang Sun, Hao Chen, Yingjun Du +2
Large vision-language models have achieved remarkable progress in visual reasoning, yet most existing systems rely on single-step or text-only reasoning, limiting their ability to…
3DGSNav: Enhancing Vision-Language Model Reasoning for Object Navigation via Active 3D Gaussian Splatting
Wancai Zheng, Hao Chen, Xianlong Lu +2
Object navigation is a core capability of embodied intelligence, enabling an agent to locate target objects in unknown environments. Recent advances in vision-language models (VLMs…
Scalable Adaptation of 3D Geometric Foundation Models via Weak Supervision from Internet Video
Zihui Gao, Ke Liu, Donny Y. Chen +5
Geometric foundation models show promise in 3D reconstruction. However, their progress is severely constrained by the scarcity of diverse, large-scale 3D annotations. While static-…
AGILE: Hand-Object Interaction Reconstruction from Video via Agentic Generation
Jin-Chuan Shi, Binhong Ye, Tao Liu +6
Reconstructing dynamic hand-object interactions from monocular videos is critical for dexterous manipulation data collection and creating realistic digital twins for robotics and V…
CausalSpatial: A Benchmark for Object-Centric Causal Spatial Reasoning
Wenxin Ma, Chenlong Wang, Ruisheng Yuan +6
Humans can look at a static scene and instantly predict what happens next -- will moving this object cause a collision? We call this ability Causal Spatial Reasoning. However, curr…
StaMo: Unsupervised Learning of Generalizable Robot Motion from Compact State Representation
Mingyu Liu, Jiuhe Shu, Hui Chen +6
A fundamental challenge in embodied intelligence is developing expressive and compact state representations for efficient world modeling and decision making. However, existing meth…
GAE: Unleashing Physical Potential of VLM with Generalizable Action Expert
Mingyu Liu, Zheng Huang, Xiaoyi Lin +6
Vision-language models demonstrate strong reasoning and planning abilities, yet grounding these predictions into precise robot actions remains a central challenge. Existing Vision-…
NoTVLA: Semantics-Preserving Robot Adaptation via Narrative Action Interfaces
Zheng Huang, Mingyu Liu, Xiaoyi Lin +9
Vision-Language-Action (VLA) models represent a pivotal advance in embodied intelligence, yet they confront critical barriers to real-world deployment, most notably catastrophic fo…
Reinforced Embodied Planning with Verifiable Reward for Real-World Robotic Manipulation
Zitong Bo, Yue Hu, Jinming Ma +7
Enabling robots to execute long-horizon manipulation tasks from free-form language instructions remains a fundamental challenge in embodied AI. While vision-language models (VLMs)…
GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Tasks
Cong Chen, Kaixiang Ji, Hao Zhong +9
Autonomous agents for long-sequence Graphical User Interface tasks are hindered by sparse rewards and the intractable credit assignment problem. To address these challenges, we int…
HieraTok: Multi-Scale Visual Tokenizer Improves Image Reconstruction and Generation
Cong Chen, Ziyuan Huang, Cheng Zou +6
In this work, we present HieraTok, a novel multi-scale Vision Transformer (ViT)-based tokenizer that overcomes the inherent limitation of modeling single-scale representations. Thi…
Tinker: Diffusion's Gift to 3D--Multi-View Consistent Editing From Sparse Inputs without Per-Scene Optimization
Canyu Zhao, Xiaoman Li, Tianjian Feng +3
We introduce Tinker, a versatile framework for high-fidelity 3D editing that operates in both one-shot and few-shot regimes without any per-scene finetuning. Unlike prior technique…
ODYSSEY: Open-World Quadrupeds Exploration and Manipulation for Long-Horizon Tasks
Kaijun Wang, Liqin Lu, Mingyu Liu +7
Language-guided long-horizon mobile manipulation has long been a grand challenge in embodied semantic reasoning, generalizable manipulation, and adaptive locomotion. Three fundamen…
Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaboration
Hao Zhong, Muzhi Zhu, Zongze Du +6
Long-horizon video-audio reasoning and fine-grained pixel understanding impose conflicting requirements on omnimodal models: dense temporal coverage demands many low-resolution fra…
ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learning
Muzhi Zhu, Hao Zhong, Canyu Zhao +9
Active vision, also known as active perception, refers to actively selecting where and how to look in order to gather task-relevant information. It is a critical component of effic…