Fisher Discriminant Triplet and Contrastive Losses for Training Siamese Networks

Abstract

Siamese neural network is a very powerful architecture for both feature extraction and metric learning. It usually consists of several networks that share weights. The Siamese concept is topology-agnostic and can use any neural network as its backbone. The two most popular loss functions for training these networks are the triplet and contrastive loss functions. In this paper, we propose two novel loss functions, named Fisher Discriminant Triplet (FDT) and Fisher Discriminant Contrastive (FDC). The former uses anchor-neighbor-distant triplets while the latter utilizes pairs of anchor-neighbor and anchor-distant samples. The FDT and FDC loss functions are designed based on the statistical formulation of the Fisher Discriminant Analysis (FDA), which is a linear subspace learning method. Our experiments on the MNIST and two challenging and publicly available histopathology datasets show the effectiveness of the proposed loss functions.

arXiv commentAccepted (to appear) in International Joint Conference on Neural Networks (IJCNN) 2020, IEEE, in IEEE World Congress on Computational Intelligence (WCCI) 2020

References in corpus (5)

• FaceNet: A Unified Embedding for Face Recognition and Clustering
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions
• Eigenvalue and Generalized Eigenvalue Problems: Tutorial
• Fisher and Kernel Fisher Discriminant Analysis: Tutorial

Cited by in corpus (9)

• Offline versus Online Triplet Mining based on Extreme Distances of Histopathology Patches
• Generalized Inter-class Loss for Gait Recognition
• VISHIEN-MAAT: Scrollytelling visualization design for explaining Siamese Neural Network concept to non-technical users
• Magnification Generalization for Histopathology Image Embedding
• Quantile-Quantile Embedding for Distribution Transformation and Manifold Embedding with Ability to Choose the Embedding Distribution
• Acceleration of Large Margin Metric Learning for Nearest Neighbor Classification Using Triplet Mining and Stratified Sampling
• Batch-Incremental Triplet Sampling for Training Triplet Networks Using Bayesian Updating Theorem
• About Explicit Variance Minimization: Training Neural Networks for Medical Imaging With Limited Data Annotations
• Vector Transport Free Riemannian LBFGS for Optimization on Symmetric Positive Definite Matrix Manifolds