6 papers · 1 filter
AdaGrad-Diff: A New Version of the Adaptive Gradient Algorithm
Matia Bojovic, Saverio Salzo, Massimiliano Pontil
Vanilla gradient methods are often highly sensitive to the choice of stepsize, which typically requires manual tuning. Adaptive methods alleviate this issue and have therefore beco…
Hyperparameter Optimization in Machine Learning
Luca Franceschi, Michele Donini, Valerio Perrone +5
Hyperparameters are configuration variables controlling the behavior of machine learning algorithms. They are ubiquitous in machine learning and artificial intelligence and the cho…
A conversion theorem and minimax optimality for continuum contextual bandits
Arya Akhavan, Karim Lounici, Massimiliano Pontil +1
We study the contextual continuum bandits problem, where the learner sequentially receives a side information vector and has to choose an action in a convex set, minimizing a funct…
Convergence Properties of Stochastic Hypergradients
Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo
Bilevel optimization problems are receiving increasing attention in machine learning as they provide a natural framework for hyperparameter optimization and meta-learning. A key st…
Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates
Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo
We study the problem of efficiently computing the derivative of the fixed-point of a parametric nondifferentiable contraction map. This problem has wide applications in machine lea…
Learning the Infinitesimal Generator of Stochastic Diffusion Processes
Vladimir R. Kostic, Karim Lounici, Helene Halconruy +2
We address data-driven learning of the infinitesimal generator of stochastic diffusion processes, essential for understanding numerical simulations of natural and physical systems.…