27 papers
Gefen: Optimized Stochastic Optimizer
Nadav Benedek, Tomer Koren, Ohad Fried
AdamW is a default optimizer for modern deep learning, but its first and second moment states add roughly two parameter-sized buffers to training memory, increasing the already sub…
The Hidden Cost of Approximation in Online Mirror Descent
Ofir Schlisselberg, Uri Sherman, Tomer Koren +1
Online mirror descent (OMD) is a fundamental algorithmic paradigm that underlies many algorithms in optimization, machine learning and sequential decision-making. The OMD iterates…
Statistical Learning from Attribution Sets
Lorne Applebaum, Robert Busa-Fekete, August Y. Chen +3
We address the problem of training conversion prediction models in advertising domains under privacy constraints, where direct links between ad clicks and conversions are unavailab…
Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity
Shira Vansover-Hager, Matan Schliserman, Ofir Schlisselberg +1
Mirror Descent (MD) extends Gradient Descent (GD) beyond Euclidean geometry and has recently reappeared as a lens for KL-regularized policy optimization in reinforcement learning a…
Near-Optimal Decentralized Stochastic Convex Optimization over Networks
Nitai Kluger, Amit Attia, Tomer Koren
We study decentralized stochastic smooth convex optimization, where workers minimize an average objective using local stochastic gradients and neighbor-only communication over…
Cost-Aware Learning
Clara Mohri, Amir Globerson, Haim Kaplan +2
We consider the problem of Cost-Aware Learning, where sampling different components of a finite-sum objective incurs different costs. The objective is to reach a target error while…