Working hard to know your neighbor's margins: Local descriptor learning loss

Abstract

We introduce a novel loss for learning local feature descriptors which is inspired by the Lowe's matching criterion for SIFT. We show that the proposed loss that maximizes the distance between the closest positive and closest negative patch in the batch is better than complex regularization methods; it works well for both shallow and deep convolution network architectures. Applying the novel loss to the L2Net CNN architecture results in a compact descriptor -- it has the same dimensionality as SIFT (128) that shows state-of-art performance in wide baseline stereo, patch verification and instance retrieval benchmarks. It is fast, computing a descriptor takes about 1 millisecond on a low-end GPU.

arXiv commentPost-NIPS-2017 update. Better hyperparameters and better results on HPatches + Brown dataset, + couple of references

References in corpus (2)

• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Learning to Compare Image Patches via Convolutional Neural Networks

Cited by in corpus (34)

• Image Matching across Wide Baselines: From Paper to Practice
• GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints
• Reference Pose Generation for Long-term Visual Localization via Learned Features and View Synthesis
• Margin Sample Mining Loss: A Deep Learning Based Method for Person Re-identification
• Recent Advances in Deep Learning Techniques for Face Recognition
• BEBLID: Boosted efficient binary local image descriptor
• ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning
• Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters
• Line as a Visual Sentence: Context-aware Line Descriptor for Visual Localization
• Leveraging Photogrammetric Mesh Models for Aerial-Ground Feature Point Matching Toward Integrated 3D Reconstruction
• PoreNet: CNN-based Pore Descriptor for High-resolution Fingerprint Recognition
• HarrisZ${}^{+}$: Harris Corner Selection for Next-Gen Image Matching Pipelines
• Vehicle Re-Identification: an Efficient Baseline Using Triplet Embedding
• Probabilistic Spatial Distribution Prior Based Attentional Keypoints Matching Network
• Learning Second-Order Attentive Context for Efficient Correspondence Pruning
• Learning to rank music tracks using triplet loss
• Towards Phytoplankton Parasite Detection Using Autoencoders
• Learning a Joint Embedding of Multiple Satellite Sensors: A Case Study for Lake Ice Monitoring
• A Performance Evaluation of Local Features for Image Based 3D Reconstruction
• AdaLAM: Revisiting Handcrafted Outlier Detection
• Learning Probabilistic Coordinate Fields for Robust Correspondences
• Visual Localization for Autonomous Driving: Mapping the Accurate Location in the City Maze
• Learning Geodesic-Aware Local Features from RGB-D Images
• RF-Net: An End-to-End Image Matching Network based on Receptive Field
• Human Motion Analysis with Deep Metric Learning
• LIFE: Lighting Invariant Flow Estimation
• Extracting Deformation-Aware Local Features by Learning to Deform
• A Survey of Feature detection methods for localisation of plain sections of Axial Brain Magnetic Resonance Imaging
• Automatic Dataset Annotation to Learn CNN Pore Description for Fingerprint Recognition
• Efficient Object Embedding for Spliced Image Retrieval
• GIMS: Image Matching System Based on Adaptive Graph Construction and Graph Neural Network
• Soft Expectation and Deep Maximization for Image Feature Detection
• TCDesc: Learning Topology Consistent Descriptors for Image Matching
• Understanding and Improving Kernel Local Descriptors