85 citations · 297 across the 54 of their papers we have counts for
56 papers
FFVO: A Feedforward Pose Decoder for Long-Horizon Visual Odometry
Meng-Li Shih, Shih-Yang Su, Yuliang Zou +7
Stable and reliable 4D spatial understanding is fundamental for autonomous driving systems. While feedforward reconstruction networks can estimate camera motion and 3D structure in…
MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planning
Letian Chen, Yiren Lu, Justin Fu +5
Multi-modal Large Language Models (MLLMs) have demonstrated remarkable capabilities in semantic understanding and common sense reasoning, making them promising candidates for solvi…
Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Driving
Jiahao Wang, Bo Sun, Yijing Bai +12
Robust training and validation of Autonomous Driving Systems (ADS) require massive, diverse datasets. Proprietary data collected by Autonomous Vehicle (AV) fleets, while high-fidel…
Scene Reconstruction as Mapping Priors for 3D Detection
Yang Fu, Yuliang Zou, Hao Xiang +8
In autonomous driving, mapping is critical for motion planning but remains an under-utilized resource for perception tasks such as 3D object detection. Maps can provide robust stru…
STELLAR: Scaling 3D Perception Large Models for Autonomous Driving
Yingwei Li, Xin Huang, Yang Liu +13
Model scaling has demonstrated remarkable success through large-scale training on diverse datasets. It remains an open question whether the same paradigm would apply to autonomous…
WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarios
Runsheng Xu, Hubert Lin, Wonseok Jeon +11
Vision-based end-to-end (E2E) driving has garnered significant interest in the research community due to its scalability and synergy with multimodal large language models (MLLMs).…
Enhanced Motion Forecasting with Plug-and-Play Multimodal Large Language Models
Katie Luo, Jingwei Ji, Tong He +4
Current autonomous driving systems rely on specialized models for perceiving and predicting motion, which demonstrate reliable performance in standard conditions. However, generali…
Drive&Gen: Co-Evaluating End-to-End Driving and Video Generation Models
Jiahao Wang, Zhenpei Yang, Yijing Bai +11
Recent advances in generative models have sparked exciting new possibilities in the field of autonomous vehicles. Specifically, video generation models are now being explored as co…
SceneDiffuser++: City-Scale Traffic Simulation via a Generative World Model
Shuhan Tan, John Lambert, Hong Jeon +6
The goal of traffic simulation is to augment a potentially limited amount of manually-driven miles that is available for testing and validation, with a much larger amount of simula…
SceneCrafter: Controllable Multi-View Driving Scene Editing
Zehao Zhu, Yuliang Zou, Chiyu Max Jiang +9
Simulation is crucial for developing and evaluating autonomous vehicle (AV) systems. Recent literature builds on a new generation of generative models to synthesize highly realisti…
S4-Driver: Scalable Self-Supervised Driving Multimodal Large Language Modelwith Spatio-Temporal Visual Representation
Yichen Xie, Runsheng Xu, Tong He +9
The latest advancements in multi-modal large language models (MLLMs) have spurred a strong renewed interest in end-to-end motion planning approaches for autonomous driving. Many en…
Scaling Laws of Motion Forecasting and Planning -- Technical Report
Mustafa Baniodeh, Kratarth Goel, Scott Ettinger +14
We study the empirical scaling laws of a family of encoder-decoder autoregressive transformer models on the task of joint motion forecasting and planning in the autonomous driving…
SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout
Chiyu Max Jiang, Yijing Bai, Andre Cornman +12
Realistic and interactive scene simulation is a key prerequisite for autonomous vehicle (AV) development. In this work, we present SceneDiffuser, a scene-level diffusion prior desi…
EMMA: End-to-End Multimodal Model for Autonomous Driving
Jyh-Jing Hwang, Runsheng Xu, Hubert Lin +11
We introduce EMMA, an End-to-end Multimodal Model for Autonomous driving. Built upon a multi-modal large language model foundation like Gemini, EMMA directly maps raw camera sensor…
PVTransformer: Point-to-Voxel Transformer for Scalable 3D Object Detection
Zhaoqi Leng, Pei Sun, Tong He +2
3D object detectors for point clouds often rely on a pooling-based PointNet to encode sparse points into grid-like voxels or pillars. In this paper, we identify that the common Poi…
MoST: Multi-modality Scene Tokenization for Motion Prediction
Norman Mu, Jingwei Ji, Zhenpei Yang +11
Many existing motion prediction approaches rely on symbolic perception outputs to generate agent trajectories, such as bounding boxes, road graph information and traffic lights. Th…
Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research
Cole Gulino, Justin Fu, Wenjie Luo +19
Simulation is an essential tool to develop and benchmark autonomous vehicle planning software in a safe and cost-effective manner. However, realistic simulation requires accurate m…
LEF: Late-to-Early Temporal Fusion for LiDAR 3D Object Detection
Tong He, Pei Sun, Zhaoqi Leng +3
We propose a late-to-early recurrent feature fusion scheme for 3D object detection using temporal LiDAR point clouds. Our main motivation is fusing object-aware latent embeddings i…
Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous Driving
Mahyar Najibi, Jingwei Ji, Yin Zhou +4
Closed-set 3D perception models trained on only a pre-defined set of object categories can be inadequate for safety critical applications such as autonomous driving where new objec…
3D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels
Zhenzhen Weng, Alexander S. Gorban, Jingwei Ji +3
Training a 3D human keypoint detector from point clouds in a supervised manner requires large volumes of high quality labels. While it is relatively easy to capture large amounts o…
MoDAR: Using Motion Forecasting for 3D Object Detection in Point Cloud Sequences
Yingwei Li, Charles R. Qi, Yin Zhou +2
Occluded and long-range objects are ubiquitous and challenging for 3D object detection. Point cloud sequence data provide unique opportunities to improve such cases, as an occluded…
MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion
Chiyu Max Jiang, Andre Cornman, Cheolho Park +3
We present MotionDiffuser, a diffusion based representation for the joint distribution of future trajectories over multiple agents. Such representation has several key advantages:…
The Waymo Open Sim Agents Challenge
Nico Montali, John Lambert, Paul Mougin +9
Simulation with realistic, interactive agents represents a key task for autonomous vehicle software development. In this work, we introduce the Waymo Open Sim Agents Challenge (WOS…
WOMD-LiDAR: Raw Sensor Dataset Benchmark for Motion Forecasting
Kan Chen, Runzhou Ge, Hang Qiu +12
Widely adopted motion forecasting datasets substitute the observed sensory inputs with higher-level abstractions such as 3D boxes and polylines. These sparse shapes are inferred th…