Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions

Abstract

We focus on the problem of predicting future states of entities in complex, real-world driving scenarios. Previous research has used low-level signals to predict short time horizons, and has not addressed how to leverage key assets relied upon heavily by industry self-driving systems: (1) large 3D perception efforts which provide highly accurate 3D states of agents with rich attributes, and (2) detailed and accurate semantic maps of the environment (lanes, traffic lights, crosswalks, etc). We present a unified representation which encodes such high-level semantic information in a spatial grid, allowing the use of deep convolutional models to fuse complex scene context. This enables learning entity-entity and entity-environment interactions with simple, feed-forward computations in each timestep within an overall temporal model of an agent's behavior. We propose different ways of modelling the future as a distribution over future states using standard supervised learning. We introduce a novel dataset providing industry-grade rich perception and semantic inputs, and empirically show we can effectively learn fundamentals of driving behavior.

arXiv commentAccepted at CVPR 2019

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Categorical Reparameterization with Gumbel-Softmax
• End to End Learning for Self-Driving Cars
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
• IntentNet: Learning to Predict Intention from Raw Sensor Data

Cited by in corpus (10)

• THOMAS: Trajectory Heatmap Output with learned Multi-Agent Sampling
• VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation
• Learning Lane Graph Representations for Motion Forecasting
• PLOP: Probabilistic poLynomial Objects trajectory Planning for autonomous driving
• Decoder Fusion RNN: Context and Interaction Aware Decoders for Trajectory Prediction
• Scene Transformer: A unified architecture for predicting multiple agent trajectories
• Improving Movement Predictions of Traffic Actors in Bird's-Eye View Models using GANs and Differentiable Trajectory Rasterization
• Advising Autonomous Cars about the Rules of the Road
• Data Driven Prediction Architecture for Autonomous Driving and its Application on Apollo Platform
• Deep Structured Reactive Planning