Learning Classifiers with Fenchel-Young Losses: Generalized Entropies, Margins, and Algorithms
arXiv:1805.09717
Abstract
This paper studies Fenchel-Young losses, a generic way to construct convex loss functions from a regularization function. We analyze their properties in depth, showing that they unify many well-known loss functions and allow to create useful new ones easily. Fenchel-Young losses constructed from a generalized entropy, including the Shannon and Tsallis entropies, induce predictive probability distributions. We formulate conditions for a generalized entropy to yield losses with a separation margin, and probability distributions with sparse support. Finally, we derive efficient algorithms, making Fenchel-Young losses appealing both in theory and practice.
arXiv commentIn proceedings of AISTATS 2019
References in corpus (7)
- SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
- Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory
- From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
- Differentiable Dynamic Programming for Structured Prediction and Attention
- A Regularized Framework for Sparse and Structured Neural Attention
- Learning with Fenchel-Young Losses
- Two-temperature logistic regression based on the Tsallis divergence