Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence
arXiv:2104.08736
Abstract
Areas under ROC (AUROC) and precision-recall curves (AUPRC) are common metrics for evaluating classification performance for imbalanced problems. Compared with AUROC, AUPRC is a more appropriate metric for highly imbalanced datasets. While stochastic optimization of AUROC has been studied extensively, principled stochastic optimization of AUPRC has been rarely explored. In this work, we propose a principled technical method to optimize AUPRC for deep learning. Our approach is based on maximizing the averaged precision (AP), which is an unbiased point estimator of AUPRC. We cast the objective into a sum of {\it dependent compositional functions} with inner functions dependent on random variables of the outer level. We propose efficient adaptive and non-adaptive stochastic algorithms named SOAP with {\it provable convergence guarantee under mild conditions} by leveraging recent advances in stochastic compositional optimization. Extensive experimental results on image and graph datasets demonstrate that our proposed method outperforms prior methods on imbalanced problems in terms of AUPRC. To the best of our knowledge, our work represents the first attempt to optimize AUPRC with provable convergence. The SOAP has been implemented in the libAUC library at~\url{https://libauc.org/}.
arXiv commentPublished on NeurIPS 2021, 24 pages, 10 figures
References in corpus (16)
- Neural Message Passing for Quantum Chemistry
- Strategies for Pre-training Graph Neural Networks
- Towards Accurate One-Stage Object Detection with AP-Loss
- AP-Loss for Accurate One-Stage Object Detection
- Training Deep Neural Networks via Direct Loss Minimization
- Optimizing Rank-based Metrics with Blackbox Differentiation
- Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization
- Finite-sum Composition Optimization via Variance Reduced Gradient Descent
- Stochastic AUC Maximization with Deep Neural Networks
- Scalable Learning of Non-Decomposable Objectives
- A Ranking-based, Balanced Loss Function Unifying Classification and Localisation in Object Detection
- Accelerated Method for Stochastic Composition Optimization with Nonsmooth Regularization
- Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks
- Improved Sample Complexity for Stochastic Compositional Variance Reduced Gradient
- Stochastically Controlled Stochastic Gradient for the Convex and Non-convex Composition problem
- An Online Method for A Class of Distributionally Robust Optimization with Non-Convex Objectives