Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoF
arXiv:2006.10178
Abstract
We solve the problem of 6-DoF localisation and 3D dense reconstruction in spatial environments as approximate Bayesian inference in a deep state-space model. Our approach leverages both learning and domain knowledge from multiple-view geometry and rigid-body dynamics. This results in an expressive predictive model of the world, often missing in current state-of-the-art visual SLAM solutions. The combination of variational inference, neural networks and a differentiable raycaster ensures that our model is amenable to end-to-end gradient-based optimisation. We evaluate our approach on realistic unmanned aerial vehicle flight data, nearing the performance of state-of-the-art visual-inertial odometry systems. We demonstrate the applicability of the model to generative prediction and planning.
arXiv commentUpdate for ICLR2021
References in corpus (17)
- Auto-Encoding Variational Bayes
- VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
- A Recurrent Latent Variable Model for Sequential Data
- Recurrent World Models Facilitate Policy Evolution
- Learning Latent Dynamics for Planning from Pixels
- A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors
- Control of Memory, Active Perception, and Action in Minecraft
- Dream to Control: Learning Behaviors by Latent Imagination
- GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations
- Neural SLAM: Learning to Explore with External Memory
- Recurrent Environment Simulators
- Efficient Model-Based Deep Reinforcement Learning with Variational State Tabulation
- Simple, Distributed, and Accelerated Probabilistic Programming
- Active Neural Localization
- Learning to Fly via Deep Model-Based Reinforcement Learning
- Approximate Bayesian inference in spatial environments
- Variational Tracking and Prediction with Generative Disentangled State-Space Models