Soft Expectation and Deep Maximization for Image Feature Detection
arXiv:2104.10291
Abstract
Central to the application of many multi-view geometry algorithms is the extraction of matching points between multiple viewpoints, enabling classical tasks such as camera pose estimation and 3D reconstruction. Many approaches that characterize these points have been proposed based on hand-tuned appearance models or data-driven learning methods. We propose Soft Expectation and Deep Maximization (SEDM), an iterative unsupervised learning process that directly optimizes the repeatability of the features by posing the problem in a similar way to expectation maximization (EM). We found convergence to be reliable and the new model to be more lighting invariant and better at localize the underlying 3D points in a scene, improving SfM quality when compared to other state of the art deep learning detectors.
arXiv comment9 pages, 3 figures, 2 tables
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
- R2D2: Repeatable and Reliable Detector and Descriptor
- LF-Net: Learning Local Features from Images
- TILDE: A Temporally Invariant Learned DEtector
- Working hard to know your neighbor's margins: Local descriptor learning loss
- DISK: Learning local features with policy gradient
- UnsuperPoint: End-to-end Unsupervised Interest Point Detector and Descriptor
- UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description without Local Correspondence Supervision
- Robust Image Retrieval-based Visual Localization using Kapture
- Visual Localization Using Semantic Segmentation and Depth Prediction