6D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inference

Abstract

We present a multimodal camera relocalization framework that captures ambiguities and uncertainties with continuous mixture models defined on the manifold of camera poses. In highly ambiguous environments, which can easily arise due to symmetries and repetitive structures in the scene, computing one plausible solution (what most state-of-the-art methods currently regress) may not be sufficient. Instead we predict multiple camera pose hypotheses as well as the respective uncertainty for each prediction. Towards this aim, we use Bingham distributions, to model the orientation of the camera pose, and a multivariate Gaussian to model the position, with an end-to-end deep neural network. By incorporating a Winner-Takes-All training scheme, we finally obtain a mixture model that is well suited for explaining ambiguities in the scene, yet does not suffer from mode collapse, a common problem with mixture density networks. We introduce a new dataset specifically designed to foster camera localization research in ambiguous environments and exhaustively evaluate our method on synthetic as well as real data on both ambiguous scenes and on non-ambiguous benchmark datasets. We plan to release our code and dataset under $\text{href}$.

Accepted for publication at ECCV 2020. Project page under https://multimodal3dvision.github.io

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• On Calibration of Modern Neural Networks
• Open3D: A Modern Library for 3D Data Processing
• RTAB-Map as an Open-Source Lidar and Visual SLAM Library for Large-Scale and Long-Term Online Operation
• Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction
• Recursive Estimation of Orientation Based on the Bingham Distribution
• Probabilistic Permutation Synchronization using the Riemannian Structure of the Birkhoff Polytope
• Bayesian Pose Graph Optimization via Bingham Distributions and Tempered Geodesic MCMC
• Directional Statistics in Machine Learning: a Brief Review
• Directional Statistics and Filtering Using libDirectional
• Scene Coordinate and Correspondence Learning for Image-Based Localization
• Learning Representations for Predicting Future Activities
• Probabilistic Regression of Rotations using Quaternion Averaging and a Deep Multi-Headed Network

Cited by in corpus (1)

• Projective Manifold Gradient Layer for Deep Rotation Regression