22 papers
Language2Drive: Do Vision-Language-Action Models Really Follow the Instructions in Autonomous Driving?
Zihan You, Zehua Fan, Wenqi Lyu +19
Recent Vision-Language-Action (VLA) models increasingly use natural-language instructions to condition autonomous driving behavior. Yet existing closed-loop benchmarks primarily me…
TacDyn-WAM: Learning Implicit Tactile Dynamics in a Heterogeneous Visuo-Tactile World Action Model
Enyi Wang, Mingxin Wang, Quan Shi +12
World action models improve robotic manipulation by conditioning actions on predicted futures, yet existing tactile variants largely inherit video-generation pipelines that reconst…
Discrete Forcing: Infusing Discrete Guidance into Continuous Denoising for Few-Step Action Experts
Jingbo Wang, Wenxuan Song, Wenhao Yu +6
Efficient action generation in vision-language-action (VLA) models requires capturing both coarse action structure and fine-grained details. Discrete action tokens provide compact…
World4Scorer: Outcome-Grounded World Modeling for Autonomous Driving
Jieyuan Pei, Meiyi Lu, Sining Ang +12
Autonomous driving requires choosing a safe and efficient plan as surrounding traffic evolves. Generate-and-select planners propose multiple trajectories and score them for executi…
StageWAM: Joint-Embedding Stage Prediction for World-Action Models in Robot Manipulation
Xiao Liu, Yuguang Yang, Xi Wang +6
Generalist robot policies aim to map multimodal observations and linguistic task instructions to actions across diverse tasks. However, existing methods typically represent the fut…
4D-WAM: Infusing Spatiotemporal Awareness into World Action Models through Trajectory Fields
Lishan Yang, Wenxuan Song, Xi Wang +14
Building on recent advances in world models, World Action Models (WAMs) jointly model video prediction and action generation. However, they typically represent videos in 2D pixel s…
JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling
Yihan Lin, Jiawei He, Shifeng Bao +6
Robust robot control benefits from explicitly modeling state transitions, but video-generation world action models (WAMs) introduce substantial deployment cost. Existing latent WAM…
Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features
Sining Ang, Yuguang Yang, Yan Wang
Large video diffusion models provide rich spatiotemporal priors for autonomous driving, but existing world-action models often inherit the cost of iterative future-video generation…
MobileWAM: Bridging World Action Models to Mobile Manipulation with Chain-of-Foresight
Zehua Fan, Junjie He, Wenxuan Song +14
World action models (WAMs) built on video generation backbones are a rising recipe for robot learning, yet remain confined to tabletop manipulation. Mobile manipulation demands sim…
ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shifts
Mingxin Wang, Bin Hu, Bin Qian +12
World Action Models (WAMs) have emerged as a promising paradigm by jointly modeling robot actions and future visual dynamics. However, their reliance on pixel-generative future sup…
Teaching Vision-Language-Action Models What to See and Where to Look
Yuguang Yang, Canyu Chen, Zhewen Tan +10
Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving. However, existing VLAs' training relies heavily on text-centric visual q…
CLOVER: Closed-Loop Value Estimation and Ranking for End-to-End Autonomous Driving Planning
Sining Ang, Yuguang Yang, Canyu Chen +1
End-to-end autonomous driving planners are commonly trained by imitating a single logged trajectory, yet evaluated by rule-based planning metrics that measure safety, feasibility,…
RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark
Huashuo Lei, Wenxuan Song, Huarui Zhang +10
Memory is a critical component of robotic intelligence, as robots must rely on past observations and actions to accomplish long-horizon tasks in partially observable environments.…
CapVector: Learning Transferable Capability Vectors in Parametric Space for Vision-Language-Action Models
Wenxuan Song, Han Zhao, Fuhao Li +7
This paper proposes a novel approach to address the challenge that pretrained VLA models often fail to effectively improve performance and reduce adaptation costs during standard s…
Cross-Sample Relational Fusion: Unifying Domain Generalization and Class-Incremental Learning
Zhen-Hao Xie, Yan Wang, Hao Sun +3
Class-Incremental Learning (CIL) requires a learning system to learn new classes while retaining previously learned knowledge. However, in real-world scenarios such as autonomous d…
SAMoE-VLA: A Scene Adaptive Mixture-of-Experts Vision-Language-Action Model for Autonomous Driving
Zihan You, Hongwei Liu, Chenxu Dang +4
Recent advances in Vision-Language-Action (VLA) models have shown promising capabilities in autonomous driving by leveraging the understanding and reasoning strengths of Large Lang…
PROSPECT: Unified Streaming Vision-Language Navigation via Semantic--Spatial Fusion and Latent Predictive Representation
Zehua Fan, Wenqi Lyu, Wenxuan Song +12
Multimodal large language models (MLLMs) have advanced zero-shot end-to-end Vision-Language Navigation (VLN), yet robust navigation requires not only semantic understanding but als…
DFM-VLA: Iterative Action Refinement for Robot Manipulation via Discrete Flow Matching
Jiayi Chen, Wenxuan Song, Jiaxin Fang +12
Vision-Language-Action (VLA) models that encode actions using a discrete tokenization scheme have been widely adopted for robotic manipulation, but existing decoding paradigms rema…
DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Driving
Chenxu Dang, Sining Ang, Yongkang Li +7
Vision-Language-Action (VLA) models for autonomous driving increasingly adopt generative planners trained with imitation learning followed by reinforcement learning. Diffusion-base…
From Representational Complementarity to Dual Systems: Synergizing VLM and Vision-Only Backbones for End-to-End Driving
Sining Ang, Yuguang Yang, Chenxu Dang +8
Vision-Language-Action (VLA) driving augments end-to-end (E2E) planning with language-enabled visual backbones, but how VLM representations differ from vision-only encoders after p…
SparseOccVLA: Bridging Occupancy and Vision-Language Models via Sparse Queries for Unified 4D Scene Understanding and Planning
Chenxu Dang, Jie Wang, Guang Li +6
In autonomous driving, Vision Language Models (VLMs) excel at high-level reasoning , whereas semantic occupancy provides fine-grained details. Despite significant progress in indiv…
SparseWorld: A Flexible, Adaptive, and Efficient 4D Occupancy World Model Powered by Sparse and Dynamic Queries
Chenxu Dang, Haiyan Liu, Jason Bao +6
Semantic occupancy has emerged as a powerful representation in world models for its ability to capture rich spatial semantics. However, most existing occupancy world models rely on…