From handcrafted to deep local features

Abstract

This paper presents an overview of the evolution of local features from handcrafted to deep-learning-based methods, followed by a discussion of several benchmarks and papers evaluating such local features. Our investigations are motivated by 3D reconstruction problems, where the precise location of the features is important. As we describe these methods, we highlight and explain the challenges of feature extraction and potential ways to overcome them. We first present handcrafted methods, followed by methods based on classical machine learning and finally we discuss methods based on deep-learning. This largely chronologically-ordered presentation will help the reader to fully understand the topic of image and region description in order to make best use of it in modern computer vision applications. In particular, understanding handcrafted methods and their motivation can help to understand modern approaches and how machine learning is used to improve the results. We also provide references to most of the relevant literature and code.

arXiv commentPreprint

References in corpus (15)

• Deep Learning in Neural Networks: An Overview
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Caffe: Convolutional Architecture for Fast Feature Embedding
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Simultaneous Feature Learning and Hash Coding with Deep Neural Networks
• TILDE: A Temporally Invariant Learned DEtector
• Learning Deep Embeddings with Histogram Loss
• Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT
• Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation
• MODS: Fast and Robust Method for Two-View Matching
• Do Convnets Learn Correspondence?
• PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors
• Learning Covariant Feature Detectors
• Large scale evaluation of local image feature detectors on homography datasets
• LATCH: Learned Arrangements of Three Patch Codes

Cited by in corpus (5)

• R2D2: Repeatable and Reliable Detector and Descriptor
• Feature Fusion of Raman Chemical Imaging and Digital Histopathology using Machine Learning for Prostate Cancer Detection
• Improving the HardNet Descriptor
• Leveraging Outdoor Webcams for Local Descriptor Learning
• CNN-based local features for navigation near an asteroid