DISK: Learning local features with policy gradient
arXiv:2006.13566
Abstract
Local feature frameworks are difficult to learn in an end-to-end fashion, due to the discreteness inherent to the selection and matching of sparse keypoints. We introduce DISK (DIScrete Keypoints), a novel method that overcomes these obstacles by leveraging principles from Reinforcement Learning (RL), optimizing end-to-end for a high number of correct feature matches. Our simple yet expressive probabilistic model lets us keep the training and inference regimes close, while maintaining good enough convergence properties to reliably train from scratch. Our features can be extracted very densely while remaining discriminative, challenging commonly held assumptions about what constitutes a good keypoint, as showcased in Fig. 1, and deliver state-of-the-art results on three public benchmarks.
arXiv commentCamera-ready version for NeurIPS 2020
References in corpus (7)
- ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
- R2D2: Repeatable and Reliable Detector and Descriptor
- LF-Net: Learning Local Features from Images
- TILDE: A Temporally Invariant Learned DEtector
- GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints
- Learning Two-View Correspondences and Geometry Using Order-Aware Network
- Matching Features without Descriptors: Implicitly Matched Interest Points
Cited by in corpus (13)
- ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning
- TopicFM: Robust and Interpretable Topic-Assisted Feature Matching
- SIFT Matching by Context Exposed
- HarrisZ: Harris Corner Selection for Next-Gen Image Matching Pipelines
- Unlocking the Use of Raw Multispectral Earth Observation Imagery for Onboard Artificial Intelligence
- Learnable Motion Coherence for Correspondence Pruning
- COTR: Correspondence Transformer for Matching Across Images
- Learning Probabilistic Coordinate Fields for Robust Correspondences
- CATs: Cost Aggregation Transformers for Visual Correspondence
- A2B: Anchor to Barycentric Coordinate for Robust Correspondence
- PRISM: PRogressive dependency maxImization for Scale-invariant image Matching
- Soft Expectation and Deep Maximization for Image Feature Detection
- Cutting-edge 3D reconstruction solutions for underwater coral reef images: A review and comparison