sigmoidF1: A Smooth F1 Score Surrogate Loss for Multilabel Classification
arXiv:2108.10566
Abstract
Multiclass multilabel classification is the task of attributing multiple labels to examples via predictions. Current models formulate a reduction of the multilabel setting into either multiple binary classifications or multiclass classification, allowing for the use of existing loss functions (sigmoid, cross-entropy, logistic, etc.). Multilabel classification reductions do not accommodate for the prediction of varying numbers of labels per example and the underlying losses are distant estimates of the performance metrics. We propose a loss function, sigmoidF1, which is an approximation of the F1 score that (1) is smooth and tractable for stochastic gradient descent, (2) naturally approximates a multilabel metric, and (3) estimates label propensities and label counts. We show that any confusion matrix metric can be formulated with a smooth surrogate. We evaluate the proposed loss function on text and image datasets, and with a variety of metrics, to account for the complexity of multilabel classification evaluation. sigmoidF1 outperforms other loss functions on one text and two image datasets and several metrics. These results show the effectiveness of using inference-time metrics as loss functions for non-trivial classification problems like multilabel classification.
arXiv commentPublished at TMLR
References in corpus (11)
- CNN: Single-label to Multi-label
- Don't Decay the Learning Rate, Increase the Batch Size
- Large-scale Multi-label Learning with Missing Labels
- Big Bird: Transformers for Longer Sequences
- ML-Net: multi-label classification of biomedical texts with deep neural networks
- A Unified View of Multi-Label Performance Measures
- Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning
- Scalable Learning of Non-Decomposable Objectives
- Top-k Multiclass SVM
- Stochastic Negative Mining for Learning with Large Output Spaces
- Simultaneous Learning of Trees and Representations for Extreme Classification and Density Estimation