34 citations · 77 across the 15 of their papers we have counts for
15 papers
Cosmos 3: Omnimodal World Models for Physical AI
NVIDIA, :, Aditi +293
We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-t…
Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
NVIDIA, :, Yan Wang +41
End-to-end architectures trained via imitation learning have advanced autonomous driving by scaling model size and data, yet performance remains brittle in safety-critical long-tai…
Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning
Zhiyu Huang, Xinshuo Weng, Maximilian Igl +5
Autonomous driving necessitates the ability to reason about future interactions between traffic agents and to make informed evaluations for planning. This paper introduces the \tex…
Promptable Closed-loop Traffic Simulation
Shuhan Tan, Boris Ivanovic, Yuxiao Chen +5
Simulation stands as a cornerstone for safe and efficient autonomous driving development. At its core a simulation system ought to produce realistic, reactive, and controllable tra…
Wolf: Dense Video Captioning with a World Summarization Framework
Boyi Li, Ligeng Zhu, Ran Tian +20
We propose Wolf, a WOrLd summarization Framework for accurate video captioning. Wolf is an automated captioning framework that adopts a mixture-of-experts approach, leveraging comp…
Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Driving
Ran Tian, Boyi Li, Xinshuo Weng +5
The autonomous driving industry is increasingly adopting end-to-end learning from sensory inputs to minimize human biases in system design. Traditional end-to-end driving models, h…
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking
Daniel Dauner, Marcel Hallgarten, Tianyu Li +9
Benchmarking vision-based driving policies is challenging. On one hand, open-loop evaluation with real data is easy, but these results do not reflect closed-loop performance. On th…
Language-Image Models with 3D Understanding
Jang Hyun Cho, Boris Ivanovic, Yulong Cao +8
Multi-modal large language models (MLLMs) have shown incredible capabilities in a variety of 2D vision and language tasks. We extend MLLMs' perceptual capabilities to ground and re…
Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Maps
Katie Z Luo, Xinshuo Weng, Yan Wang +5
Autonomous driving has traditionally relied heavily on costly and labor-intensive High Definition (HD) maps, hindering scalability. In contrast, Standard Definition (SD) maps are m…
EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision
Jiawei Yang, Boris Ivanovic, Or Litany +8
We present EmerNeRF, a simple yet powerful approach for learning spatial-temporal representations of dynamic driving scenes. Grounded in neural fields, EmerNeRF simultaneously capt…
Language Conditioned Traffic Generation
Shuhan Tan, Boris Ivanovic, Xinshuo Weng +2
Simulation forms the backbone of modern self-driving development. Simulators help develop, test, and improve driving systems without putting humans, vehicles, or their environment…
Task-Aware Risk Estimation of Perception Failures for Autonomous Vehicles
Pasquale Antonante, Sushant Veer, Karen Leung +3
Safety and performance are key enablers for autonomous driving: on the one hand we want our autonomous vehicles (AVs) to be safe, while at the same time their performance (e.g., co…
Tree-structured Policy Planning with Learned Behavior Models
Yuxiao Chen, Peter Karkus, Boris Ivanovic +2
Autonomous vehicles (AVs) need to reason about the multimodal behavior of neighboring agents while planning their own motion. Many existing trajectory planners seek a single trajec…
Robust Trajectory Prediction against Adversarial Attacks
Yulong Cao, Danfei Xu, Xinshuo Weng +4
Trajectory prediction using deep neural networks (DNNs) is an essential component of autonomous driving (AD) systems. However, these methods are vulnerable to adversarial attacks,…
Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking
Jinkun Cao, Jiangmiao Pang, Xinshuo Weng +2
Kalman filter (KF) based methods for multi-object tracking (MOT) make an assumption that objects move linearly. While this assumption is acceptable for very short periods of occlus…