Attention-Aware Age-Agnostic Visual Place Recognition

Abstract

A cross-domain visual place recognition (VPR) task is proposed in this work, i.e., matching images of the same architectures depicted in different domains. VPR is commonly treated as an image retrieval task, where a query image from an unknown location is matched with relevant instances from geo-tagged gallery database. Different from conventional VPR settings where the query images and gallery images come from the same domain, we propose a more common but challenging setup where the query images are collected under a new unseen condition. The two domains involved in this work are contemporary street view images of Amsterdam from the Mapillary dataset (source domain) and historical images of the same city from Beeldbank dataset (target domain). We tailored an age-invariant feature learning CNN that can focus on domain invariant objects and learn to match images based on a weakly supervised ranking loss. We propose an attention aggregation module that is robust to domain discrepancy between the train and the test data. Further, a multi-kernel maximum mean discrepancy (MK-MMD) domain adaptation loss is adopted to improve the cross-domain ranking performance. Both attention and adaptation modules are unsupervised while the ranking loss uses weak supervision. Visual inspection shows that the attention module focuses on built forms while the dramatically changing environment are less weighed. Our proposed CNN achieves state of the art results (99% accuracy) on the single-domain VPR task and 20% accuracy at its best on the cross-domain VPR task, revealing the difficulty of age-invariant VPR.

arXiv commentPresented at ICCV WORKSHOP ON E-HERITAGE 2019, Seoul, South Korea

References in corpus (9)

• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Attention-Based Models for Speech Recognition
• Particular object retrieval with integral max-pooling of CNN activations
• Attention-based Pyramid Aggregation Network for Visual Place Recognition
• Deep CNN-based Multi-task Learning for Open-Set Recognition
• Discriminative Label Consistent Domain Adaptation
• Deep Attentional Structured Representation Learning for Visual Recognition
• Semantically-Aware Attentive Neural Embeddings for Image-based Visual Localization