Diverse Trajectory Forecasting with Determinantal Point Processes

Abstract

The ability to forecast a set of likely yet diverse possible future behaviors of an agent (e.g., future trajectories of a pedestrian) is essential for safety-critical perception systems (e.g., autonomous vehicles). In particular, a set of possible future behaviors generated by the system must be diverse to account for all possible outcomes in order to take necessary safety precautions. It is not sufficient to maintain a set of the most likely future outcomes because the set may only contain perturbations of a single outcome. While generative models such as variational autoencoders (VAEs) have been shown to be a powerful tool for learning a distribution over future trajectories, randomly drawn samples from the learned implicit likelihood model may not be diverse -- the likelihood model is derived from the training data distribution and the samples will concentrate around the major mode that has most data. In this work, we propose to learn a diversity sampling function (DSF) that generates a diverse and likely set of future trajectories. The DSF maps forecasting context features to a set of latent codes which can be decoded by a generative model (e.g., VAE) into a set of diverse trajectory samples. Concretely, the process of identifying the diverse set of samples is posed as a parameter estimation of the DSF. To learn the parameters of the DSF, the diversity of the trajectory samples is evaluated by a diversity loss based on a determinantal point process (DPP). Gradient descent is performed over the DSF parameters, which in turn move the latent codes of the sample set to find an optimal diverse and likely set of trajectories. Our method is a novel application of DPPs to optimize a set of items (trajectories) in continuous space. We demonstrate the diversity of the trajectories produced by our approach on both low-dimensional 2D trajectory data and high-dimensional human motion data.

ICLR 2020

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Determinantal point processes for machine learning
• Expectation-Maximization for Learning Determinantal Point Processes
• Learning 3D Human Dynamics from Video
• Ego-Pose Estimation and Forecasting as Real-Time PD Control
• GDPP: Learning Diverse Generations Using Determinantal Point Process

Cited by in corpus (16)

• LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving
• A Novel Graph based Trajectory Predictor with Pseudo Oracle
• An information-theoretic perspective on intrinsic motivation in reinforcement learning: a survey
• Diverse Human Motion Prediction Guided by Multi-Level Spatial-Temporal Anchors
• Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
• Probabilistic Time Series Forecasting with Structured Shape and Temporal Diversity
• DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets
• PTP: Parallelized Tracking and Prediction with Graph Neural Networks and Diversity Sampling
• DLow: Diversifying Latent Flows for Diverse Human Motion Prediction
• Generating Smooth Pose Sequences for Diverse Human Motion Prediction
• Generative Hybrid Representations for Activity Forecasting with No-Regret Learning
• DiversityGAN: Diversity-Aware Vehicle Motion Prediction via Latent Semantic Sampling
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors
• Exploring Dynamic Context for Multi-path Trajectory Prediction
• From Recognition to Prediction: Analysis of Human Action and Trajectory Prediction in Video
• Deep Sequence Learning for Video Anticipation: From Discrete and Deterministic to Continuous and Stochastic