A Symmetric Loss Perspective of Reliable Machine Learning

Abstract

When minimizing the empirical risk in binary classification, it is a common practice to replace the zero-one loss with a surrogate loss to make the learning objective feasible to optimize. Examples of well-known surrogate losses for binary classification include the logistic loss, hinge loss, and sigmoid loss. It is known that the choice of a surrogate loss can highly influence the performance of the trained classifier and therefore it should be carefully chosen. Recently, surrogate losses that satisfy a certain symmetric condition (aka., symmetric losses) have demonstrated their usefulness in learning from corrupted labels. In this article, we provide an overview of symmetric losses and their applications. First, we review how a symmetric loss can yield robust classification from corrupted labels in balanced error rate (BER) minimization and area under the receiver operating characteristic curve (AUC) maximization. Then, we demonstrate how the robust AUC maximization method can benefit natural language processing in the problem where we want to learn only from relevant keywords and unlabeled documents. Finally, we conclude this article by discussing future directions, including potential applications of symmetric losses for reliable machine learning and the design of non-symmetric losses that can benefit from the symmetric condition.

arXiv commentInvited article preprint

References in corpus (29)

• Deep Learning in Neural Networks: An Overview
• Recent Advances in Open Set Recognition: A Survey
• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• A Crowdsourcing Framework for On-Device Federated Learning
• Noise Tolerance under Risk Minimization
• Making Risk Minimization Tolerant to Label Noise
• Composite Binary Losses
• Normalized Loss Functions for Deep Learning with Noisy Labels
• A Survey of Label-noise Representation Learning: Past, Present and Future
• Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning
• On Symmetric Losses for Learning from Corrupted Labels
• Semi-Supervised AUC Optimization based on Positive-Unlabeled Learning
• On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
• Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences
• Mitigating Overfitting in Supervised Classification from Two Unlabeled Datasets: A Consistent Risk Correction Approach
• Exact Exponent in Optimal Rates for Crowdsourcing
• A survey on domain adaptation theory: learning bounds and theoretical guarantees
• Analysis of Minimax Error Rate for Crowdsourcing and Its Application to Worker Clustering Model
• Integrating Semantic Knowledge to Tackle Zero-shot Text Classification
• Multi-class Classification without Multi-class Labels
• Imitation Learning from Imperfect Demonstration
• Binary Classification with Karmic, Threshold-Quasi-Concave Metrics
• Classification with Rejection Based on Cost-sensitive Classification
• On the Calibration of Multiclass Classification with Rejection
• Robust Imitation Learning from Noisy Demonstrations
• Sparse Robust Classification via the Kernel Mean
• When Optimizing $f$-divergence is Robust with Label Noise
• Revisiting Sample Selection Approach to Positive-Unlabeled Learning: Turning Unlabeled Data into Positive rather than Negative
• FIND: Human-in-the-Loop Debugging Deep Text Classifiers