A Survey on Metric Learning for Feature Vectors and Structured Data
arXiv:1306.6709
Abstract
The need for appropriate ways to measure the distance or similarity between data is ubiquitous in machine learning, pattern recognition and data mining, but handcrafting such good metrics for specific problems is generally difficult. This has led to the emergence of metric learning, which aims at automatically learning a metric from data and has attracted a lot of interest in machine learning and related fields for the past ten years. This survey paper proposes a systematic review of the metric learning literature, highlighting the pros and cons of each approach. We pay particular attention to Mahalanobis distance metric learning, a well-studied and successful framework, but additionally present a wide range of methods that have recently emerged as powerful alternatives, including nonlinear metric learning, similarity learning and local metric learning. Recent trends and extensions, such as semi-supervised metric learning, metric learning for histogram data and the derivation of generalization guarantees, are also covered. Finally, this survey addresses metric learning for structured data, in particular edit distance learning, and attempts to give an overview of the remaining challenges in metric learning for the years to come.
arXiv commentTechnical report, 59 pages. Changes in v2: fixed typos and improved presentation. Changes in v3: fixed typos. Changes in v4: fixed typos and new methods
References in corpus (17)
- Marginalized Denoising Autoencoders for Domain Adaptation
- Domain Adaptation: Learning Bounds and Algorithms
- Robustness and Generalization for Metric Learning
- Positive Semidefinite Metric Learning with Boosting
- Parametric Local Metric Learning for Nearest Neighbor Classification
- The Power of Asymmetry in Binary Hashing
- Distance Metric Learning for Kernel Machines
- Similarity Learning for Provably Accurate Sparse Linear Classification
- On the Generalization Ability of Online Learning Algorithms for Pairwise Loss Functions
- Large-Margin Metric Learning for Partitioning Problems
- Information-theoretic Semi-supervised Metric Learning via Entropy Regularization
- Generalization Bounds for Metric and Similarity Learning
- Robust Metric Learning by Smooth Optimization
- Adaptive Regularization for Weight Matrices
- Online Learning with Pairwise Loss Functions
- Supervised Metric Learning with Generalization Guarantees
- Learning Discriminative Metrics via Generative Models and Kernel Learning
Cited by in corpus (105)
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- A Decade Survey of Content Based Image Retrieval using Deep Learning
- Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization
- Few-shot classification in Named Entity Recognition Task
- Perspectives on individual animal identification from biology and computer vision
- Transferring Knowledge Fragments for Learning Distance Metric from A Heterogeneous Domain
- Robustness and Generalization for Metric Learning
- Attentive Recurrent Comparators
- Optimizing Rank-based Metrics with Blackbox Differentiation
- Few-Shot Learning Through an Information Retrieval Lens
- Ground Metric Learning
- Geometric Mean Metric Learning
- Gaussian Prototypical Networks for Few-Shot Learning on Omniglot
- Modeling and Recognition of Smart Grid Faults by a Combined Approach of Dissimilarity Learning and One-Class Classification
- Unseen Class Discovery in Open-world Classification
- Heterogeneous Multi-task Metric Learning across Multiple Domains
- Deep Weighted Averaging Classifiers
- Distributed stochastic optimization via matrix exponential learning
- Subspace Alignment For Domain Adaptation
- Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning
- Online convex optimization and no-regret learning: Algorithms, guarantees and applications
- Representing high throughput expression profiles via perturbation barcodes reveals compound targets
- Federated Continual Learning: Concepts, Challenges, and Solutions
- Relative Geometry-Aware Siamese Neural Network for 6DOF Camera Relocalization
- Online but Accurate Inference for Latent Variable Models with Local Gibbs Sampling
- Two Simple Ways to Learn Individual Fairness Metrics from Data
- Similarity Learning for High-Dimensional Sparse Data
- Supervised LogEuclidean Metric Learning for Symmetric Positive Definite Matrices
- Deep-RBF Networks Revisited: Robust Classification with Rejection
- Sample complexity of learning Mahalanobis distance metrics
- An Empirical Study on Learning Fairness Metrics for COMPAS Data with Human Supervision
- Guided Unsupervised Learning by Subaperture Decomposition for Ocean SAR Image Retrieval
- Transfer Metric Learning: Algorithms, Applications and Outlooks
- Sparse Compositional Metric Learning
- Escaping the Curse of Dimensionality in Similarity Learning: Efficient Frank-Wolfe Algorithm and Generalization Bounds
- Locality Aware Appearance Metric for Multi-Target Multi-Camera Tracking
- A Tutorial on Distance Metric Learning: Mathematical Foundations, Algorithms, Experimental Analysis, Prospects and Challenges (with Appendices on Mathematical Background and Detailed Algorithms Explanation)
- Deep learning for nano-photonic materials -- The solution to everything!?
- Enhancing Bayesian model updating in structural health monitoring via learnable mappings
- Towards Self-Adaptive Metric Learning On the Fly
- High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning
- Towards Human Body-Part Learning for Model-Free Gait Recognition
- Context-Aware Visual Compatibility Prediction
- Unit Commitment using Nearest Neighbor as a Short-Term Proxy
- Extended playing techniques: The next milestone in musical instrument recognition
- AffinityNet: semi-supervised few-shot learning for disease type prediction
- Learning Local Invariant Mahalanobis Distances
- Sparse online relative similarity learning
- Actively Learning Hemimetrics with Applications to Eliciting User Preferences
- Doubly Robust Data-Driven Distributionally Robust Optimization
- Metric Learning-based Generative Adversarial Network
- Optimal link prediction with matrix logistic regression
- A contribution to Optimal Transport on incomparable spaces
- Learning Cost Functions for Optimal Transport
- Projective Latent Interventions for Understanding and Fine-tuning Classifiers
- The Mapillary Traffic Sign Dataset for Detection and Classification on a Global Scale
- Exploit Bounding Box Annotations for Multi-label Object Recognition
- A Projection Method for Metric-Constrained Optimization
- Adaptive Image Stream Classification via Convolutional Neural Network with Intrinsic Similarity Metrics
- Calibrated simplex-mapping classification
- Dynamic Input Structure and Network Assembly for Few-Shot Learning
- Metric Learning from Imbalanced Data
- Similarity Kernel and Clustering via Random Projection Forests
- Embeddings and Representation Learning for Structured Data
- Nonparametric Online Regression while Learning the Metric
- metricDTW: local distance metric learning in Dynamic Time Warping
- Iterated Support Vector Machines for Distance Metric Learning
- Multi-view Common Component Discriminant Analysis for Cross-view Classification
- Local Distance Metric Learning for Nearest Neighbor Algorithm
- Multi-view metric learning for multi-instance image classification
- Representation Learning with Weighted Inner Product for Universal Approximation of General Similarities
- Maximum Margin Clustering for State Decomposition of Metastable Systems
- Interpolated Discretized Embedding of Single Vectors and Vector Pairs for Classification, Metric Learning and Distance Approximation
- Learning Condensed and Aligned Features for Unsupervised Domain Adaptation Using Label Propagation
- Dynamic Metric Learning from Pairwise Comparisons
- Metric Learning with Dynamically Generated Pairwise Constraints for Ear Recognition
- Similarity Function Tracking using Pairwise Comparisons
- A Riemannian Primal-dual Algorithm Based on Proximal Operator and its Application in Metric Learning
- Visual-Interactive Similarity Search for Complex Objects by Example of Soccer Player Analysis
- Instance-Based Neural Dependency Parsing
- Feasibility Based Large Margin Nearest Neighbor Metric Learning
- Interpretable Distance Metric Learning for Handwritten Chinese Character Recognition
- Nonstationary Distance Metric Learning
- Local Higher-Order Statistics (LHS) describing images with statistics of local non-binarized pixel patterns
- Scalable Nonlinear Embeddings for Semantic Category-based Image Retrieval
- Nonlinear Metric Learning for kNN and SVMs through Geometric Transformations
- Expert-guided Regularization via Distance Metric Learning
- Constructing Binary Descriptors with a Stochastic Hill Climbing Search
- End-to-End Data Visualization by Metric Learning and Coordinate Transformation
- Cooperative Bi-path Metric for Few-shot Learning
- Lifelong Metric Learning
- Deep Embedding using Bayesian Risk Minimization with Application to Sketch Recognition
- Collaborative Translational Metric Learning
- Approximate Eigenvalue Decompositions of Linear Transformations with a Few Householder Reflectors
- Distance metric learning based on structural neighborhoods for dimensionality reduction and classification performance improvement
- Hyperlink Regression via Bregman Divergence
- Secure Metric Learning via Differential Pairwise Privacy
- A Fast and Easy Regression Technique for k-NN Classification Without Using Negative Pairs
- Variational learning across domains with triplet information
- Nonlinear Metric Learning through Geodesic Interpolation within Lie Groups
- Interpretable Locally Adaptive Nearest Neighbors
- Multiple Metric Learning for Structured Data
- Towards Sharper Utility Bounds for Differentially Private Pairwise Learning
- Semi-Supervised Nonlinear Distance Metric Learning via Forests of Max-Margin Cluster Hierarchies
- Towards ECDSA key derivation from deep embeddings for novel Blockchain applications