Localising In Complex Scenes Using Balanced Adversarial Adaptation
arXiv:2011.04122
Abstract
Domain adaptation and generative modelling have collectively mitigated the expensive nature of data collection and labelling by leveraging the rich abundance of accurate, labelled data in simulation environments. In this work, we study the performance gap that exists between representations optimised for localisation on simulation environments and the application of such representations in a real-world setting. Our method exploits the shared geometric similarities between simulation and real-world environments whilst maintaining invariance towards visual discrepancies. This is achieved by optimising a representation extractor to project both simulated and real representations into a shared representation space. Our method uses a symmetrical adversarial approach which encourages the representation extractor to conceal the domain that features are extracted from and simultaneously preserves robust attributes between source and target domains that are beneficial for localisation. We evaluate our method by adapting representations optimised for indoor Habitat simulated environments (Matterport3D and Replica) to a real-world indoor environment (Active Vision Dataset), showing that it compares favourably against fully-supervised approaches.
arXiv commentAccepted at 3DV 2020
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Deep Visual Domain Adaptation: A Survey
- DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
- Domain Separation Networks
- Unsupervised Cross-Domain Image Generation
- The Replica Dataset: A Digital Replica of Indoor Spaces
- Matterport3D: Learning from RGB-D Data in Indoor Environments
- Neural SLAM: Learning to Explore with External Memory
- Neural Map: Structured Memory for Deep Reinforcement Learning
- One-Shot Adaptation of Supervised Deep Convolutional Models