FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras
arXiv:2104.10490
Abstract
Driving requires interacting with road agents and predicting their future behaviour in order to navigate safely. We present FIERY: a probabilistic future prediction model in bird's-eye view from monocular cameras. Our model predicts future instance segmentation and motion of dynamic agents that can be transformed into non-parametric future trajectories. Our approach combines the perception, sensor fusion and prediction components of a traditional autonomous driving stack by estimating bird's-eye-view prediction directly from surround RGB monocular camera inputs. FIERY learns to model the inherent stochastic nature of the future solely from camera driving data in an end-to-end manner, without relying on HD maps, and predicts multimodal future trajectories. We show that our model outperforms previous prediction baselines on the NuScenes and Lyft datasets. The code and trained models are available at https://github.com/wayveai/fiery.
arXiv commentICCV 2021
References in corpus (11)
- Deep Continuous Fusion for Multi-Sensor 3D Object Detection
- Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
- Cross-view Semantic Segmentation for Sensing Surroundings
- IntentNet: Learning to Predict Intention from Raw Sensor Data
- Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networks
- Multiple Futures Prediction
- FISHING Net: Future Inference of Semantic Heatmaps In Grids
- BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloud
- Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses
- On the Benefit of Adversarial Training for Monocular Depth Estimation
- LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion