Publications (16)
MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction
Yuning Chai, Benjamin Sapp, Mayank Bansal +1
Predicting human behavior is a difficult and crucial task required for motion planning. It is challenging in large part due to the highly uncertain and multi-modal set of possible…
TNT: Target-driveN Trajectory Prediction
Hang Zhao, Jiyang Gao, Tian Lan +9
Predicting the future behavior of moving agents is essential for real world applications. It is challenging as the intent of the agent and the corresponding behavior is unknown and…
Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions
Joey Hong, Benjamin Sapp, James Philbin
We focus on the problem of predicting future states of entities in complex, real-world driving scenarios. Previous research has used low-level signals to predict short time horizon…
JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving
Wenjie Luo, Cheolho Park, Andre Cornman +2
We propose JFP, a Joint Future Prediction model that can learn to generate accurate and consistent multi-agent future trajectories. For this task, many different methods have been…
Scaling Laws of Motion Forecasting and Planning -- Technical Report
Mustafa Baniodeh, Kratarth Goel, Scott Ettinger +14
We study the empirical scaling laws of a family of encoder-decoder autoregressive transformer models on the task of joint motion forecasting and planning in the autonomous driving…
Narrowing the Coordinate-frame Gap in Behavior Prediction Models: Distillation for Efficient and Accurate Scene-centric Motion Forecasting
DiJia Su, Bertrand Douillard, Rami Al-Rfou +2
Behavior prediction models have proliferated in recent years, especially in the popular real-world robotics application of autonomous driving, where representing the distribution o…
Structured Prediction Cascades
David Weiss, Benjamin Sapp, Ben Taskar
Structured prediction tasks pose a fundamental trade-off between the need for model complexity to increase predictive power and the limited computational resources for inference in…
The Science Performance of JWST as Characterized in Commissioning
Jane Rigby, Marshall Perrin, Michael McElwain +623
This paper characterizes the actual science performance of the James Webb Space Telescope (JWST), as determined from the six month commissioning period. We summarize the performanc…
Identifying Driver Interactions via Conditional Behavior Prediction
Ekaterina Tolstaya, Reza Mahjourian, Carlton Downey +3
Interactive driving scenarios, such as lane changes, merges and unprotected turns, are some of the most challenging situations for autonomous driving. Planning in interactive scena…
MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction
Balakrishnan Varadarajan, Ahmed Hefny, Avikalp Srivastava +8
Predicting the future behavior of road users is one of the most challenging and important problems in autonomous driving. Applying deep learning to this problem requires fusing het…
The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition
Jonathan Krause, Benjamin Sapp, Andrew Howard +5
Current approaches for fine-grained recognition do the following: First, recruit experts to annotate a dataset of images, optionally also collecting more structured data in the for…
MotionLM: Multi-Agent Motion Forecasting as Language Modeling
Ari Seff, Brian Cera, Dian Chen +6
Reliable forecasting of the future behavior of road agents is a critical component to safe planning in autonomous vehicles. Here, we represent continuous trajectories as sequences…
Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research
Cole Gulino, Justin Fu, Wenjie Luo +19
Simulation is an essential tool to develop and benchmark autonomous vehicle planning software in a safe and cost-effective manner. However, realistic simulation requires accurate m…
Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios
Yiren Lu, Justin Fu, George Tucker +9
Imitation learning (IL) is a simple and powerful way to use high-quality human driving data, which can be collected at scale, to produce human-like behavior. However, policies base…
Wayformer: Motion Forecasting via Simple & Efficient Attention Networks
Nigamaa Nayakanti, Rami Al-Rfou, Aurick Zhou +3
Motion forecasting for autonomous driving is a challenging task because complex driving scenarios result in a heterogeneous mix of static and dynamic inputs. It is an open problem…
EMMA: End-to-End Multimodal Model for Autonomous Driving
Jyh-Jing Hwang, Runsheng Xu, Hubert Lin +11
We introduce EMMA, an End-to-end Multimodal Model for Autonomous driving. Built upon a multi-modal large language model foundation like Gemini, EMMA directly maps raw camera sensor…