CAR-Net: Clairvoyant Attentive Recurrent Network

Abstract

We present an interpretable framework for path prediction that leverages dependencies between agents' behaviors and their spatial navigation environment. We exploit two sources of information: the past motion trajectory of the agent of interest and a wide top-view image of the navigation scene. We propose a Clairvoyant Attentive Recurrent Network (CAR-Net) that learns where to look in a large image of the scene when solving the path prediction task. Our method can attend to any area, or combination of areas, within the raw image (e.g., road intersections) when predicting the trajectory of the agent. This allows us to visualize fine-grained semantic elements of navigation scenes that influence the prediction of trajectories. To study the impact of space on agents' trajectories, we build a new dataset made of top-view images of hundreds of scenes (Formula One racing tracks) where agents' behaviors are heavily influenced by known areas in the images (e.g., upcoming turns). CAR-Net successfully attends to these salient regions. Additionally, CAR-Net reaches state-of-the-art accuracy on the standard trajectory forecasting benchmark, Stanford Drone Dataset (SDD). Finally, we show CAR-Net's ability to generalize to unseen scenes.

arXiv commentThe 2nd and 3rd authors contributed equally

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Social Force Model for Pedestrian Dynamics
• Recurrent Models of Visual Attention
• DRAW: A Recurrent Neural Network For Image Generation
• Multiple Object Recognition with Visual Attention
• Action Recognition using Visual Attention
• Image Captioning with Semantic Attention
• DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
• Forecasting Social Navigation in Crowded Complex Scenes
• To Go or Not To Go? A Near Unsupervised Learning Approach For Robot Navigation
• Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies
• GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimation
• Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition
• VUNet: Dynamic Scene View Synthesis for Traversability Estimation using an RGB Camera

Cited by in corpus (6)

• Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks
• SafeCritic: Collision-Aware Trajectory Prediction
• A Transferable Pedestrian Motion Prediction Model for Intersections with Different Geometries
• Relational Action Forecasting
• Multi-Agent Tensor Fusion for Contextual Trajectory Prediction
• Convolutions for Spatial Interaction Modeling