19 papers
World Action Critic (WAC) Model: Joint Foresight and Criticism for Policy Learning
Xiangteng Zhang, Haoyuan Liu, Yang Guan +7
World models have shown considerable promise in sequential decision-making by anticipating the consequences of actions and providing predictive foresight for policy learning. Howev…
Past, Future, All at Once: Mitigating Stability-Plasticity Dilemma via Post-hoc JANUS Rectification
Zhilong Zheng, Letian Tao, Yang Guan +7
Fine-tuning foundation models on new tasks inevitably suffer from catastrophic forgetting. While existing works attempt to mitigate this on the basis of parameter-efficient fine-tu…
Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation
Shiqi Liu, Zeyu He, Letian Tao +9
On-policy distillation (OPD) has emerged as an effective approach for large language model post-training, yet existing objectives face a trade-off between objective fidelity and op…
Latent Two-Sample Testing for Fair Autonomous Vehicle Road Evaluation
Qiujing Lu, Xuanhan Wang, Guanghong Jia +5
With the rapid advancement of autonomous vehicle (AV) systems, fast and reliable iteration through road testing has become increasingly critical. However, changes in testing enviro…
FAST: A Framework for Aligned Sampling and Training in Parallel Reinforcement Learning for Autonomous Driving
Bonan Wang, Letian Tao, Bin Shuai +7
Deep reinforcement learning is pivotal for closed-loop autonomous driving yet remains constrained by severe bottlenecks in sampling efficiency. Standard parallel sampling mitigates…
Teaching Vision-Language-Action Models What to See and Where to Look
Yuguang Yang, Canyu Chen, Zhewen Tan +10
Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving. However, existing VLAs' training relies heavily on text-centric visual q…
DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
Chen Shi, Jinrui Xu, Shaoshuai Shi +3
Pretrained foundation models have become an important basis for end-to-end autonomous driving. In contrast to vision-language models pretrained primarily on static image-text pairs…
Unveiling the Surprising Efficacy of Navigation Understanding in End-to-End Autonomous Driving
Zhihua Hua, Junli Wang, Pengfei LI +6
Global navigation information and local scene understanding are two crucial components of autonomous driving systems. However, our experimental results indicate that many end-to-en…
STAPO: Stabilizing Reinforcement Learning for LLMs by Silencing Rare Spurious Tokens
Shiqi Liu, Zeyu He, Guojian Zhan +10
Reinforcement Learning (RL) has significantly improved large language model reasoning, but existing RL fine-tuning methods rely heavily on heuristic techniques such as entropy regu…
Semore: VLM-guided Enhanced Semantic Motion Representations for Visual Reinforcement Learning
Wentao Wang, Chunyang Liu, Kehua Sheng +2
The growing exploration of Large Language Models (LLM) and Vision-Language Models (VLM) has opened avenues for enhancing the effectiveness of reinforcement learning (RL). However,…
UniSplat: Unified Spatio-Temporal Fusion via 3D Latent Scaffolds for Dynamic Driving Scene Reconstruction
Chen Shi, Shaoshuai Shi, Xiaoyang Lyu +4
Feed-forward 3D reconstruction for autonomous driving has advanced rapidly, yet existing methods struggle with the joint challenges of sparse, non-overlapping camera views and comp…
PriorFusion: Unified Integration of Priors for Robust Road Perception in Autonomous Driving
Xuewei Tang, Mengmeng Yang, Tuopu Wen +7
With the growing interest in autonomous driving, there is an increasing demand for accurate and reliable road perception technologies. In complex environments without high-definiti…
DriveX: Omni Scene Modeling for Learning Generalizable World Knowledge in Autonomous Driving
Chen Shi, Shaoshuai Shi, Kehua Sheng +2
Data-driven learning has advanced autonomous driving, yet task-specific models struggle with out-of-distribution scenarios due to their narrow optimization objectives and reliance…
An End-to-End Learning-Based Multi-Sensor Fusion for Autonomous Vehicle Localization
Changhong Lin, Jiarong Lin, Zhiqiang Sui +4
Multi-sensor fusion is essential for autonomous vehicle localization, as it is capable of integrating data from various sources for enhanced accuracy and reliability. The accuracy…
DiffMap: Enhancing Map Segmentation with Map Prior Using Diffusion Model
Peijin Jia, Tuopu Wen, Ziang Luo +9
Constructing high-definition (HD) maps is a crucial requirement for enabling autonomous driving. In recent years, several map segmentation algorithms have been developed to address…
Transferring CLIP's Knowledge into Zero-Shot Point Cloud Semantic Segmentation
Yuanbin Wang, Shaofei Huang, Yulu Gao +5
Traditional 3D segmentation methods can only recognize a fixed range of classes that appear in the training set, which limits their application in real-world scenarios due to the l…
MultiSPANS: A Multi-range Spatial-Temporal Transformer Network for Traffic Forecast via Structural Entropy Optimization
Dongcheng Zou, Senzhang Wang, Xuefeng Li +5
Traffic forecasting is a complex multivariate time-series regression task of paramount importance for traffic management and planning. However, existing approaches often struggle t…
LiON: Learning Point-wise Abstaining Penalty for LiDAR Outlier DetectioN Using Diverse Synthetic Data
Shaocong Xu, Pengfei Li, Qianpu Sun +11
LiDAR-based semantic scene understanding is an important module in the modern autonomous driving perception stack. However, identifying outlier points in a LiDAR point cloud is cha…
Delving into Shape-aware Zero-shot Semantic Segmentation
Xinyu Liu, Beiwen Tian, Zhen Wang +5
Thanks to the impressive progress of large-scale vision-language pretraining, recent recognition models can classify arbitrary objects in a zero-shot and open-set manner, with a su…