180 citations · 519 across the 23 of their papers we have counts for
25 papers
PhysWAM: Physically Consistent World Action Model for Autonomous Driving
Dhruv Parikh, Fengcheng Yu, Quankai Gao +11
World-action models (WAMs) jointly predict how a scene will evolve and how an agent should act, however joint generation alone does not necessarily impose a shared geometric constr…
DreamStream: Towards Policy-Oriented Generative Simulation for End-to-End Driving
Ziyang Leng, Sicheng Mo, Seth Z. Zhao +4
Faithfully evaluating end-to-end driving policies in simulation requires observations that are not merely photo-realistic, but preserve the scene features a policy relies on to mak…
NavSafe-: Benchmarking Closed-Loop Driving Safety in Photorealistic Environments
Yuxin Bao, Hongwei Ruan, Luobin Wang +7
End-to-end (E2E) driving policies have advanced rapidly on open-loop (OL) benchmarks, yet OL evaluation cannot reveal whether a policy can withstand compounding errors, recover fro…
The Impact of Class Uncertainty Propagation in Perception-Based Motion Planning
Jibran Iqbal Shah, Andrei Ivanovic, Kelly Zhu +4
Autonomous vehicles (AVs) are being increasingly deployed in urban environments. In order to operate safely and reliably, AVs need to account for the inherent uncertainty associate…
NeuralRemaster: Phase-Preserving Diffusion for Structure-Aligned Generation
Yu Zeng, Charles Ochoa, Mingyuan Zhou +3
Standard diffusion corrupts data using Gaussian noise whose Fourier coefficients have random magnitudes and random phases. While effective for unconditional or text-to-image genera…
CARFF: Conditional Auto-encoded Radiance Field for 3D Scene Forecasting
Jiezhi Yang, Khushi Desai, Charles Packer +4
We propose CARFF, a method for predicting future 3D scenes given past observations. Our method maps 2D ego-centric images to a distribution over plausible 3D latent scene configura…
Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning?
Gunshi Gupta, Tim G. J. Rudner, Rowan Thomas McAllister +2
Causal confusion is a phenomenon where an agent learns a policy that reflects imperfect spurious correlations in the data. Such a policy may falsely appear to be optimal during tra…
Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research
Cole Gulino, Justin Fu, Wenjie Luo +19
Simulation is an essential tool to develop and benchmark autonomous vehicle planning software in a safe and cost-effective manner. However, realistic simulation requires accurate m…
In-Distribution Barrier Functions: Self-Supervised Policy Filters that Avoid Out-of-Distribution States
Fernando Castañeda, Haruki Nishimura, Rowan McAllister +2
Learning-based control approaches have shown great promise in performing complex tasks directly from high-dimensional perception data for real robotic systems. Nonetheless, the lea…
RAP: Risk-Aware Prediction for Robust Planning
Haruki Nishimura, Jean Mercat, Blake Wulfe +2
Robust planning in interactive scenarios requires predicting the uncertain future to make risk-aware decisions. Unfortunately, due to long-tail safety-critical events, the risk is…
Sociotechnical Specification for the Broader Impacts of Autonomous Vehicles
Thomas Krendl Gilbert, Aaron J. Snoswell, Michael Dennis +2
Autonomous Vehicles (AVs) will have a transformative impact on society. Beyond the local safety and efficiency of individual vehicles, these effects will also change how people int…
Control-Aware Prediction Objectives for Autonomous Driving
Rowan McAllister, Blake Wulfe, Jean Mercat +3
Autonomous vehicle software is typically structured as a modular pipeline of individual components (e.g., perception, prediction, and planning) to help separate concerns into inter…
Dynamics-Aware Comparison of Learned Reward Functions
Blake Wulfe, Ashwin Balakrishna, Logan Ellis +3
The ability to learn reward functions plays an important role in enabling the deployment of intelligent agents in the real world. However, comparing reward functions, for example a…
Contingencies from Observations: Tractable Contingency Planning with Learned Behavior Models
Nicholas Rhinehart, Jeff He, Charles Packer +4
Humans have a remarkable ability to make decisions by accurately reasoning about future events, including the future behaviors and states of mind of other agents. Consider driving…
Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty
Boris Ivanovic, Kuan-Hui Lee, Pavel Tokmakov +4
Reasoning about the future behavior of other agents is critical to safe robot navigation. The multiplicity of plausible futures is further amplified by the uncertainty inherent to…
Outcome-Driven Reinforcement Learning via Variational Inference
Tim G. J. Rudner, Vitchyr H. Pong, Rowan McAllister +2
While reinforcement learning algorithms provide automated acquisition of optimal policies, practical application of such methods requires a number of design decisions, such as manu…
Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?
Angelos Filos, Panagiotis Tigas, Rowan McAllister +3
Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment, typically leading to arbitrary deductions and poorly-informed decisions. In pr…
Learning Invariant Representations for Reinforcement Learning without Reconstruction
Amy Zhang, Rowan McAllister, Roberto Calandra +2
We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstructio…
Model-Based Meta-Reinforcement Learning for Flight with Suspended Payloads
Suneel Belkhale, Rachel Li, Gregory Kahn +3
Transporting suspended payloads is challenging for autonomous aerial vehicles because the payload can cause significant and unpredictable changes to the robot's dynamics. These cha…
Safety Augmented Value Estimation from Demonstrations (SAVED): Safe Deep Model-Based RL for Sparse Cost Robotic Tasks
Brijen Thananjeyan, Ashwin Balakrishna, Ugo Rosolia +6
Reinforcement learning (RL) for robotics is challenging due to the difficulty in hand-engineering a dense cost function, which can lead to unintended behavior, and dynamical uncert…
PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settings
Nicholas Rhinehart, Rowan McAllister, Kris Kitani +1
For autonomous vehicles (AVs) to behave appropriately on roads populated by human-driven vehicles, they must be able to reason about the uncertain intentions and decisions of other…
Robustness to Out-of-Distribution Inputs via Task-Aware Generative Uncertainty
Rowan McAllister, Gregory Kahn, Jeff Clune +1
Deep learning provides a powerful tool for machine perception when the observations resemble the training data. However, real-world robotic systems must react intelligently to thei…
Deep Imitative Models for Flexible Inference, Planning, and Control
Nicholas Rhinehart, Rowan McAllister, Sergey Levine
Imitation Learning (IL) is an appealing approach to learn desirable autonomous behavior. However, directing IL to achieve arbitrary goals is difficult. In contrast, planning-based…
Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
Kurtland Chua, Roberto Calandra, Rowan McAllister +1
Model-based reinforcement learning (RL) algorithms can attain excellent sample efficiency, but often lag behind the best model-free algorithms in terms of asymptotic performance. T…