On the alpha-loss Landscape in the Logistic Model
arXiv:2006.12406
Abstract
We analyze the optimization landscape of a recently introduced tunable class of loss functions called -loss, , in the logistic model. This family encapsulates the exponential loss (), the log-loss (), and the 0-1 loss () and contains compelling properties that enable the practitioner to discern among a host of operating conditions relevant to emerging learning methods. Specifically, we study the evolution of the optimization landscape of -loss with respect to using tools drawn from the study of strictly-locally-quasi-convex functions in addition to geometric techniques. We interpret these results in terms of optimization complexity via normalized gradient descent.
arXiv comment5 pages, appeared in ISIT 2020. arXiv admin note: text overlap with arXiv:1906.02314
References in corpus (6)
- Focal Loss for Dense Object Detection
- Exploring the Landscape of Spatial Robustness
- The loss surface of deep and wide neural networks
- Beyond Convexity: Stochastic Quasi-Convex Optimization
- Understanding the Loss Surface of Neural Networks for Binary Classification
- A Tunable Measure for Information Leakage