Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions

Abstract

Recent innovations in training deep convolutional neural network (ConvNet) models have motivated the design of new methods to automatically learn local image descriptors. The latest deep ConvNets proposed for this task consist of a siamese network that is trained by penalising misclassification of pairs of local image patches. Current results from machine learning show that replacing this siamese by a triplet network can improve the classification accuracy in several problems, but this has yet to be demonstrated for local image descriptor learning. Moreover, current siamese and triplet networks have been trained with stochastic gradient descent that computes the gradient from individual pairs or triplets of local image patches, which can make them prone to overfitting. In this paper, we first propose the use of triplet networks for the problem of local image descriptor learning. Furthermore, we also propose the use of a global loss that minimises the overall classification error in the training set, which can improve the generalisation capability of the model. Using the UBC benchmark dataset for comparing local image descriptors, we show that the triplet network produces a more accurate embedding than the siamese network in terms of the UBC dataset errors. Moreover, we also demonstrate that a combination of the triplet and global losses produces the best embedding in the field, using this triplet network. Finally, we also show that the use of the central-surround siamese network trained with the global loss produces the best result of the field on the UBC dataset. Pre-trained models are available online at https://github.com/vijaykbg/deep-patchmatch

IEEE Conference on Computer Vision and Pattern Recognition 2016 (CVPR 2016)

References in corpus (1)

• Learning to Compare Image Patches via Convolutional Neural Networks

Cited by in corpus (20)

• A Decade Survey of Content Based Image Retrieval using Deep Learning
• A Survey on Deep Learning Techniques for Stereo-based Depth Estimation
• GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints
• Outlier Exposure with Confidence Control for Out-of-Distribution Detection
• A Differential Approach for Gaze Estimation
• Smart Mining for Deep Metric Learning
• Fisher Discriminant Triplet and Contrastive Losses for Training Siamese Networks
• DeepPoint3D: Learning Discriminative Local Descriptors using Deep Metric Learning on 3D Point Clouds
• Feature-based groupwise registration of historical aerial images to present-day ortho-photo maps
• Audio-visual cross-modality knowledge transfer for machine learning-based in-situ monitoring in laser additive manufacturing
• Estimating and abstracting the 3D structure of bones using neural networks on X-ray (2D) images
• A Theoretically Sound Upper Bound on the Triplet Loss for Improving the Efficiency of Deep Distance Metric Learning
• Self-Supervised Video Representation Learning With Odd-One-Out Networks
• Generative Model with Coordinate Metric Learning for Object Recognition Based on 3D Models
• Semi-supervised learning with Bidirectional GANs
• Visual Geometric Skill Inference by Watching Human Demonstration
• Accurate Deep Representation Quantization with Gradient Snapping Layer for Similarity Search
• When Relation Networks meet GANs: Relation GANs with Triplet Loss
• Learning Transformation-Aware Embeddings for Image Forensics
• On Learning Density Aware Embeddings