How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change

Abstract

Direct visual localization has recently enjoyed a resurgence in popularity with the increasing availability of cheap mobile computing power. The competitive accuracy and robustness of these algorithms compared to state-of-the-art feature-based methods, as well as their natural ability to yield dense maps, makes them an appealing choice for a variety of mobile robotics applications. However, direct methods remain brittle in the face of appearance change due to their underlying assumption of photometric consistency, which is commonly violated in practice. In this paper, we propose to mitigate this problem by training deep convolutional encoder-decoder models to transform images of a scene such that they correspond to a previously-seen canonical appearance. We validate our method in multiple environments and illumination conditions using high-fidelity synthetic RGB-D datasets, and integrate the trained models into a direct visual localization pipeline, yielding improvements in visual odometry (VO) accuracy through time-varying illumination conditions, as well as improved metric relocalization performance under illumination change, where conventional methods normally fail. We further provide a preliminary investigation of transfer learning from synthetic to real environments in a localization context. An open-source implementation of our method using PyTorch is available at https://github.com/utiasSTARS/cat-net.

arXiv commentIn IEEE Robotics and Automation Letters (RA-L) and presented at the IEEE International Conference on Robotics and Automation (ICRA'18), Brisbane, Australia, May 21-25, 2018

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
• End-to-End Training of Deep Visuomotor Policies
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
• Backprop KF: Learning Discriminative Deterministic State Estimators
• Reducing Drift in Visual Odometry by Inferring Sun Direction Using a Bayesian Convolutional Neural Network

Cited by in corpus (6)

• DPC-Net: Deep Pose Correction for Visual Localization
• Learned Camera Gain and Exposure Control for Improved Visual Feature Detection and Matching
• SPP-Net: Deep Absolute Pose Regression with Synthetic Views
• Learning Matchable Image Transformations for Long-term Metric Visual Localization
• Evolution of Visual Odometry Techniques
• Fast Rigid 3D Registration Solution: A Simple Method Free of SVD and Eigen-Decomposition