An Asymmetric Contrastive Loss for Handling Imbalanced Datasets

Abstract

Contrastive learning is a representation learning method performed by contrasting a sample to other similar samples so that they are brought closely together, forming clusters in the feature space. The learning process is typically conducted using a two-stage training architecture, and it utilizes the contrastive loss (CL) for its feature learning. Contrastive learning has been shown to be quite successful in handling imbalanced datasets, in which some classes are overrepresented while some others are underrepresented. However, previous studies have not specifically modified CL for imbalanced datasets. In this work, we introduce an asymmetric version of CL, referred to as ACL, in order to directly address the problem of class imbalance. In addition, we propose the asymmetric focal contrastive loss (AFCL) as a further generalization of both ACL and focal contrastive loss (FCL). Results on the FMNIST and ISIC 2018 imbalanced datasets show that AFCL is capable of outperforming CL and FCL in terms of both weighted and unweighted classification accuracies. In the appendix, we provide a full axiomatic treatment on entropy, along with complete proofs.

arXiv comment15 pages, 5 figures

References in corpus (10)

• A Simple Framework for Contrastive Learning of Visual Representations
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions
• Learning deep representations by mutual information estimation and maximization
• Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• CUSBoost: Cluster-based Under-sampling with Boosting for Imbalanced Classification
• Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning
• GenSample: A Genetic Algorithm for Oversampling in Imbalanced Datasets
• VOS: a Method for Variational Oversampling of Imbalanced Data