SoftTriple Loss: Deep Metric Learning Without Triplet Sampling

Abstract

Distance metric learning (DML) is to learn the embeddings where examples from the same class are closer than examples from different classes. It can be cast as an optimization problem with triplet constraints. Due to the vast number of triplet constraints, a sampling strategy is essential for DML. With the tremendous success of deep learning in classifications, it has been applied for DML. When learning embeddings with deep neural networks (DNNs), only a mini-batch of data is available at each iteration. The set of triplet constraints has to be sampled within the mini-batch. Since a mini-batch cannot capture the neighbors in the original set well, it makes the learned embeddings sub-optimal. On the contrary, optimizing SoftMax loss, which is a classification loss, with DNN shows a superior performance in certain DML tasks. It inspires us to investigate the formulation of SoftMax. Our analysis shows that SoftMax loss is equivalent to a smoothed triplet loss where each class has a single center. In real-world data, one class can contain several local clusters rather than a single one, e.g., birds of different poses. Therefore, we propose the SoftTriple loss to extend the SoftMax loss with multiple centers for each class. Compared with conventional deep metric learning algorithms, optimizing SoftTriple loss can learn the embeddings without the sampling phase by mildly increasing the size of the last fully connected layer. Experiments on the benchmark fine-grained data sets demonstrate the effectiveness of the proposed loss function. Code is available at https://github.com/idstcv/SoftTriple

arXiv commentaccepted by ICCV'19

References in corpus (1)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Cited by in corpus (26)

• Revisiting Training Strategies and Generalization Performance in Deep Metric Learning
• It Takes Two to Tango: Mixup for Deep Metric Learning
• Camera-aware Proxies for Unsupervised Person Re-Identification
• Proxy Anchor Loss for Deep Metric Learning
• NPT-Loss: A Metric Loss with Implicit Mining for Face Recognition
• Unifying Specialist Image Embedding into Universal Image Embedding
• Quantile-Quantile Embedding for Distribution Transformation and Manifold Embedding with Ability to Choose the Embedding Distribution
• Ranked List Loss for Deep Metric Learning
• S2SD: Simultaneous Similarity-based Self-Distillation for Deep Metric Learning
• Fashion-IQ 2020 Challenge 2nd Place Team's Solution
• Bag of Tricks and A Strong baseline for Image Copy Detection
• Hierarchical Proxy-based Loss for Deep Metric Learning
• Learning Condition Invariant Features for Retrieval-Based Localization from 1M Images
• Alleviating the Incompatibility between Cross Entropy Loss and Episode Training for Few-shot Skin Disease Classification
• Universal Model for Multi-Domain Medical Image Retrieval
• Uniform Priors for Data-Efficient Transfer
• Deep Cross-modal Hashing via Margin-dynamic-softmax Loss
• Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning
• Rethinking Deep Contrastive Learning with Embedding Memory
• Learning Discriminative Representations for Fine-Grained Diabetic Retinopathy Grading
• Color Variants Identification in Fashion e-commerce via Contrastive Self-Supervised Representation Learning
• Diversified Mutual Learning for Deep Metric Learning
• VS-Net: Voting with Segmentation for Visual Localization
• Deep Metric Learning with Locality Sensitive Angular Loss for Self-Correcting Source Separation of Neural Spiking Signals
• Dynamic Metric Learning: Towards a Scalable Metric Space to Accommodate Multiple Semantic Scales
• Improving Sample Efficiency with Normalized RBF Kernels