32 papers · 1 filter
Language2Drive: Do Vision-Language-Action Models Really Follow the Instructions in Autonomous Driving?
Zihan You, Zehua Fan, Wenqi Lyu +19
Recent Vision-Language-Action (VLA) models increasingly use natural-language instructions to condition autonomous driving behavior. Yet existing closed-loop benchmarks primarily me…
Efficient Multi-Modal Planning with Reward-Guided Preference Optimization for Autonomous Driving
Chenglin Chen, Lujia Wang, Xinhu Zheng +2
Safe and efficient trajectory planning is essential in autonomous driving. However, existing end-to-end approaches often fall short in both computational efficiency and safety guar…
A Topological Representation with Object-Path Graphs for Open-Vocabulary Instance Navigation
Linwei Zheng, Daojie Peng, Bingtao Wang +2
Vision-language navigation requires embodied agents to navigate environments using natural language instructions and visual observations. Existing approaches typically decompose na…
DreamTrajectory: Trajectory-Guided Action Generation with World Model Alignment for Mobile Manipulation
Zheng Yang, Wenjie Zhang, Xiangyu Chen +9
Mobile manipulation requires a robot to coordinate base and arm motion under continuously changing viewpoints and contact conditions, within an action space far larger than that of…
4D-WAM: Infusing Spatiotemporal Awareness into World Action Models through Trajectory Fields
Lishan Yang, Wenxuan Song, Xi Wang +14
Building on recent advances in world models, World Action Models (WAMs) jointly model video prediction and action generation. However, they typically represent videos in 2D pixel s…
SG-WAM: Text-Grounded and Spatial-aware Semantic Guidance for World-Action Models
Junjie He, Junfeng Li, Zhide Zhong +9
World-Action Models (WAMs) have emerged as a promising paradigm for robotic manipulation. However, most existing WAMs generate future videos and actions by relying mainly on visual…
Is Forward Prediction Enough? Physical State Grounding for JEPA World Models
Haodong Yan, Jiaguan Zhu, Mingyuan Jia +12
Learning structured and control-relevant latent representations remains a key challenge for world models. Recent JEPA-based world models learn action-conditioned predictive latent…
DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation
Junfeng Li, Junjie He, Zhide Zhong +12
Vision-Language-Action (VLA) models have become a powerful paradigm for robot manipulation, but training a single generalist policy for heterogeneous robot embodiments remains an o…
Diffusion-based 4D Trajectory Prediction and Distributed Control for UAV Swarms
Tianshun Li, Hongliang Lu, Haoang Li +1
Accurate 4D trajectory prediction and closed-loop tracking are essential for Unmanned Aerial Vehicle (UAV) swarms to achieve safe and efficient operations in complex low-altitude e…
SEDualVLN: A Spatially-Enhanced Dual-System Framework for Vision-Language Navigation
Jingzhi Huang, Junkai Huang, Wenxuan Song +6
Vision-Language Navigation (VLN) approaches have currently followed two primary paradigms: the end-to-end Vision-Language Model (VLM) policy fine-tuned on navigation trajectories t…
Efficient Feature-Free Initialization for Monocular Visual-Inertial Systems Using a Feed-Forward 3D Model
Yuantai Zhang, Jiaqi Yang, Huajian Zeng +5
Fast and reliable initialization is critical for monocular visual-inertial navigation systems (VINS), as it establishes the starting conditions for subsequent state estimation. Des…
HCSG: Human-Centric Semantic-Geometric Reasoning for Vision-Language Navigation
Haoxuan Xu, Tianfu Li, Wenbo Chen +7
VLN has achieved remarkable progress by scaling data and model capacity. However, the assumption of a static environment breaks down in real-world indoor scenarios, where robots in…
RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark
Huashuo Lei, Wenxuan Song, Huarui Zhang +10
Memory is a critical component of robotic intelligence, as robots must rely on past observations and actions to accomplish long-horizon tasks in partially observable environments.…
Robust Energy-Aware Routing for Air-Ground Cooperative Multi-UAV Delivery in Wind-Uncertain Environments
Tianshun Li, Hongliang Lu, Yanggang Sheng +3
Ensuring energy feasibility under wind uncertainty is critical for the safety and reliability of UAV delivery missions. In realistic truck-drone logistics systems, UAVs must delive…
Fast-dVLA: Accelerating Discrete Diffusion VLA to Real-Time Performance
Wenxuan Song, Jiayi Chen, Shuai Chen +8
This paper proposes a novel approach to address the challenge that pretrained VLA models often fail to effectively improve performance and reduce adaptation costs during standard s…
VLA-OPD: Bridging Offline SFT and Online RL for Vision-Language-Action Models via On-Policy Distillation
Zhide Zhong, Haodong Yan, Junfeng Li +3
Although pre-trained Vision-Language-Action (VLA) models exhibit impressive generalization in robotic manipulation, post-training remains crucial to ensure reliable performance dur…
DyGeoVLN: Infusing Dynamic Geometry Foundation Model into Vision-Language Navigation
Xiangchen Liu, Hanghan Zheng, Jeil Jeong +5
Vision-language Navigation (VLN) requires an agent to understand visual observations and language instructions to navigate in unseen environments. Most existing approaches rely on…
VAMPO: Policy Optimization for Improving Visual Dynamics in Video Action Models
Zirui Ge, Pengxiang Ding, Baohua Yin +16
Video action models are an appealing foundation for Vision--Language--Action systems because they can learn visual dynamics from large-scale video data and transfer this knowledge…
DFM-VLA: Iterative Action Refinement for Robot Manipulation via Discrete Flow Matching
Jiayi Chen, Wenxuan Song, Jiaxin Fang +12
Vision-Language-Action (VLA) models that encode actions using a discrete tokenization scheme have been widely adopted for robotic manipulation, but existing decoding paradigms rema…
Enhancing Vision-Language Navigation with Multimodal Event Knowledge from Real-World Indoor Tour Videos
Haoxuan Xu, Tianfu Li, Wenbo Chen +4
Vision-Language Navigation (VLN) agents often struggle with long-horizon reasoning in unseen environments, particularly when facing ambiguous, coarse-grained instructions. While re…
FRAPPE: Infusing World Modeling into Generalist Policies via Multiple Future Representation Alignment
Han Zhao, Jingbo Wang, Wenxuan Song +5
Enabling VLA models to predict environmental dynamics, known as world modeling, has been recognized as essential for improving robotic reasoning and generalization. However, curren…
The RoboSense Challenge: Sense Anything, Navigate Anywhere, Adapt Across Platforms
Lingdong Kong, Shaoyuan Xie, Zeying Gong +135
Autonomous systems are increasingly deployed in open and dynamic environments -- from city streets to aerial and indoor spaces -- where perception models must remain reliable under…
Embodied Robot Manipulation in the Era of Foundation Models: Planning and Learning Perspectives
Shuanghao Bai, Wenxuan Song, Jiayi Chen +14
Recent advances in vision, language, and multimodal learning have significantly accelerated progress in robotic foundation models, with robotic manipulation remaining one of the mo…
Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Process
Jiayi Chen, Wenxuan Song, Pengxiang Ding +5
Vision-language-action (VLA) models aim to understand natural language instructions and visual observations and to execute corresponding actions as an embodied agent. Recent work i…