Reducing Drift in Visual Odometry by Inferring Sun Direction Using a Bayesian Convolutional Neural Network

Abstract

We present a method to incorporate global orientation information from the sun into a visual odometry pipeline using only the existing image stream, where the sun is typically not visible. We leverage recent advances in Bayesian Convolutional Neural Networks to train and implement a sun detection model that infers a three-dimensional sun direction vector from a single RGB image. Crucially, our method also computes a principled uncertainty associated with each prediction, using a Monte Carlo dropout scheme. We incorporate this uncertainty into a sliding window stereo visual odometry pipeline where accurate uncertainty estimates are critical for optimal data fusion. Our Bayesian sun detection model achieves a median error of approximately 12 degrees on the KITTI odometry benchmark training set, and yields improvements of up to 42% in translational ARMSE and 32% in rotational ARMSE compared to standard VO. An open source implementation of our Bayesian CNN sun estimator (Sun-BCNN) using Caffe is available at https://github. com/utiasSTARS/sun-bcnn-vo

arXiv commentIn Proceedings of the IEEE International Conference on Robotics and Automation (ICRA'17), Singapore, May 29-Jun. 3, 2017

References in corpus (5)

• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
• PROBE-GK: Predictive Robust Estimation using Generalized Kernels
• Improving the Accuracy of Stereo Visual Odometry Using Visual Illumination Estimation
• Noise Models in Feature-based Stereo Visual Odometry

Cited by in corpus (5)

• DPC-Net: Deep Pose Correction for Visual Localization
• How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change
• Self-Supervised Deep Pose Corrections for Robust Visual Odometry
• Shadow Transfer: Single Image Relighting For Urban Road Scenes
• ALPaCA vs. GP-based Prior Learning: A Comparison between two Bayesian Meta-Learning Algorithms