Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning

Abstract

A family of loss functions built on pair-based computation have been proposed in the literature which provide a myriad of solutions for deep metric learning. In this paper, we provide a general weighting framework for understanding recent pair-based loss functions. Our contributions are three-fold: (1) we establish a General Pair Weighting (GPW) framework, which casts the sampling problem of deep metric learning into a unified view of pair weighting through gradient analysis, providing a powerful tool for understanding recent pair-based loss functions; (2) we show that with GPW, various existing pair-based methods can be compared and discussed comprehensively, with clear differences and key limitations identified; (3) we propose a new loss called multi-similarity loss (MS loss) under the GPW, which is implemented in two iterative steps (i.e., mining and weighting). This allows it to fully consider three similarities for pair weighting, providing a more principled approach for collecting and weighting informative pairs. Finally, the proposed MS loss obtains new state-of-the-art performance on four image retrieval benchmarks, where it outperforms the most recent approaches, such as ABE\cite{Kim_2018_ECCV} and HTL by a large margin: 60.6% to 65.7% on CUB200, and 80.9% to 88.0% on In-Shop Clothes Retrieval dataset at Recall@1. Code is available at https://github.com/MalongTech/research-ms-loss.

arXiv commentAccepted CVPR 2019, rewrite main method to be more clear

References in corpus (2)

• Learning Deep Embeddings with Histogram Loss
• Deep Metric Learning for Practical Person Re-Identification

Cited by in corpus (14)

• Circle Loss: A Unified Perspective of Pair Similarity Optimization
• Deep Metric Learning-based Image Retrieval System for Chest Radiograph and its Clinical Applications in COVID-19
• Domain Generalization via Optimal Transport with Metric Similarity Learning
• Unsupervised Deep Metric Learning via Auxiliary Rotation Loss
• Ranked List Loss for Deep Metric Learning
• Towards Visually Explaining Similarity Models
• Calibrated neighborhood aware confidence measure for deep metric learning
• Fashion-IQ 2020 Challenge 2nd Place Team's Solution
• Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning
• A New Training Framework for Deep Neural Network
• MetricOpt: Learning to Optimize Black-Box Evaluation Metrics
• Toward Interpretability of Dual-Encoder Models for Dialogue Response Suggestions
• Color Variants Identification in Fashion e-commerce via Contrastive Self-Supervised Representation Learning
• Dynamic Metric Learning: Towards a Scalable Metric Space to Accommodate Multiple Semantic Scales