Improving the HardNet Descriptor
arXiv:2007.09699
Abstract
In the thesis we consider the problem of local feature descriptor learning for wide baseline stereo focusing on the HardNet descriptor, which is close to state-of-the-art. AMOS Patches dataset is introduced, which improves robustness to illumination and appearance changes. It is based on registered images from selected cameras from the AMOS dataset. We provide recommendations on the patch dataset creation process and evaluate HardNet trained on data of different modalities. We also introduce a dataset combination and reduction methods, that allow comparable performance on a significantly smaller dataset. HardNet8, consistently outperforming the original HardNet, benefits from the architectural choices made: connectivity pattern, final pooling, receptive field, CNN building blocks found by manual or automatic search algorithms -- DARTS. We show impact of overlooked hyperparameters such as batch size and length of training on the descriptor quality. PCA dimensionality reduction further boosts performance and also reduces memory footprint. Finally, the insights gained lead to two HardNet8 descriptors: one performing well on a variety of benchmarks -- HPatches, AMOS Patches and IMW Phototourism, the other is optimized for IMW Phototourism.
arXiv commentThe thesis was supervised by Dmytro Mishkin. Many pieces of advice came from Jiří Matas
References in corpus (14)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- DARTS: Differentiable Architecture Search
- fastai: A Layered API for Deep Learning
- An Analysis of Deep Neural Network Models for Practical Applications
- A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
- R2D2: Repeatable and Reliable Detector and Descriptor
- TILDE: A Temporally Invariant Learned DEtector
- Systematic evaluation of CNN advances on the ImageNet
- GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints
- MODS: Fast and Robust Method for Two-View Matching
- From handcrafted to deep local features
- A Large Dataset for Improving Patch Matching
- To Learn or Not to Learn: Visual Localization from Essential Matrices
- Leveraging Outdoor Webcams for Local Descriptor Learning