9 papers
AVERT-VLN: Abstention-aware Visual Error Recovery and Training for Vision-and-Language Navigation
Minrui Liu, Jingke Wang, Yuehao Huang +4
Deploying vision-and-language navigation (VLN) agents in unseen environments remains challenging because unfamiliar layouts and visual conditions can cause execution to go off trac…
Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout
Haozhuang Chi, Daosheng Qiu, Hao Su +4
Safe L2/L3 driving automation requires anticipating human-in-the-loop reactions during shared-control transitions. While most driving world models forecast the external environment…
Risk-Aware Selective Multimodal Driver Monitoring with Driver-State World Modeling
Daosheng Qiu, Haozhuang Chi, Hao Su +4
Continuous driver monitoring in automated vehicles requires low-latency inference while avoiding unsafe decisions under uncertain driver states. Large vision-language models provid…
SAGE-Nav: Leveraging LLM Planning and Alignment Fusion for Hierarchical Scene Graph-Guided Navigation
Hao Su, Yuehao Huang, Yukai Ma +2
Object-Goal Navigation (ObjNav) requires embodied agents to autonomously locate specified targets using only egocentric visual observations. Existing monolithic methods struggle wi…
DriveStack-VLA: Render-Teacher Alignment for BEV-Based DeepStack Vision-Language-Action Model
Jingke Wang, Zhenru Zhao, Shuangming Lei +8
Vision-Language-Action driving models convert a pretrained Vision-Language Model into a driving policy, allowing them to use world knowledge and follow language guidances. However,…
Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning
Adrià López Escoriza, Nicklas Hansen, Stone Tao +2
Long-horizon tasks in robotic manipulation present significant challenges in reinforcement learning (RL) due to the difficulty of designing dense reward functions and effectively e…
Responsive Noise-Relaying Diffusion Policy: Responsive and Efficient Visuomotor Control
Zhuoqun Chen, Xiu Yuan, Tongzhou Mu +1
Imitation learning is an efficient method for teaching robots a variety of tasks. Diffusion Policy, which uses a conditional denoising diffusion process to generate actions, has de…
When Should We Prefer State-to-Visual DAgger Over Visual Reinforcement Learning?
Tongzhou Mu, Zhaoyang Li, Stanisław Wiktor Strzelecki +4
Learning policies from high-dimensional visual inputs, such as pixels and point clouds, is crucial in various applications. Visual reinforcement learning is a promising approach th…
Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model
Xiu Yuan, Tongzhou Mu, Stone Tao +3
Recent advancements in robot learning have used imitation learning with large models and extensive demonstrations to develop effective policies. However, these models are often lim…