LoFTR: Detector-Free Local Feature Matching with Transformers

Abstract

We present a novel method for local image feature matching. Instead of performing image feature detection, description, and matching sequentially, we propose to first establish pixel-wise dense matches at a coarse level and later refine the good matches at a fine level. In contrast to dense methods that use a cost volume to search correspondences, we use self and cross attention layers in Transformer to obtain feature descriptors that are conditioned on both images. The global receptive field provided by Transformer enables our method to produce dense matches in low-texture areas, where feature detectors usually struggle to produce repeatable interest points. The experiments on indoor and outdoor datasets show that LoFTR outperforms state-of-the-art methods by a large margin. LoFTR also ranks first on two public benchmarks of visual localization among the published methods.

Accepted to CVPR 2021. Project page: https://zju3dv.github.io/loftr/

References in corpus (8)

• R2D2: Repeatable and Reliable Detector and Descriptor
• Reformer: The Efficient Transformer
• Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
• Reference Pose Generation for Long-term Visual Localization via Learned Features and View Synthesis
• Learning Two-View Correspondences and Geometry Using Order-Aware Network
• GIFT: Learning Transformation-Invariant Dense Visual Descriptors via Group CNNs
• ContextDesc: Local Descriptor Augmentation with Cross-Modality Context
• Robust Image Retrieval-based Visual Localization using Kapture

Cited by in corpus (10)

• TopicFM: Robust and Interpretable Topic-Assisted Feature Matching
• HarrisZ${}^{+}$: Harris Corner Selection for Next-Gen Image Matching Pipelines
• Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey
• COTR: Correspondence Transformer for Matching Across Images
• PosDiffNet: Positional Neural Diffusion for Point Cloud Registration in a Large Field of View with Perturbations
• CATs: Cost Aggregation Transformers for Visual Correspondence
• End-to-end Evaluation of Practical Video Analytics Systems for Face Detection and Recognition
• Nonlinear Intensity Underwater Sonar Image Matching Method Based on Phase Information and Deep Convolution Features
• Homography augumented momentum constrastive learning for SAR image retrieval
• Pose Refinement with Joint Optimization of Visual Points and Lines