106 citations · 525 across the 44 of their papers we have counts for
6 papers · 1 filter
Online Learning: A Modern Introduction Using Convex Optimization
Francesco Orabona
In this book, I introduce the concepts of online learning through a modern view based on convex optimization. Here, online learning refers to the framework of regret minimization u…
Parameter-Free Locally Differentially Private Stochastic Subgradient Descent
Kwang-Sung Jun, Francesco Orabona
We consider the problem of minimizing a convex risk with stochastic subgradients guaranteeing -locally differentially private (-LDP). While it has been shown that stochastic…
Kernel Truncated Randomized Ridge Regression: Optimal Rates and Low Noise Acceleration
Kwang-Sung Jun, Ashok Cutkosky, Francesco Orabona
In this paper, we consider the nonparametric least square regression in a Reproducing Kernel Hilbert Space (RKHS). We propose a new randomized algorithm that has optimal generaliza…
Momentum-Based Variance Reduction in Non-Convex SGD
Ashok Cutkosky, Francesco Orabona
Variance reduction has emerged in recent years as a strong competitor to stochastic gradient descent in non-convex problems, providing the first algorithms to improve upon the conv…
Parameter-Free Online Convex Optimization with Sub-Exponential Noise
Kwang-Sung Jun, Francesco Orabona
We consider the problem of unconstrained online convex optimization (OCO) with sub-exponential noise, a strictly more general problem than the standard OCO. In this setting, the le…
Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization
Zhenxun Zhuang, Ashok Cutkosky, Francesco Orabona
Stochastic Gradient Descent (SGD) has played a central role in machine learning. However, it requires a carefully hand-picked stepsize for fast convergence, which is notoriously te…