11 papers
AdaGrad does not adapt to Hölder-smoothness for composite objectives
Matia Bojovic, Saverio Salzo, Massimiliano Pontil
We exhibit a simple deterministic one-dimensional convex composite optimization problem for which AdaGrad scheme does not achieve the classical convergence rate $\mathcal{O}(n^{-(1…
Bilevel learning
Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo +1
Bilevel learning refers to machine learning problems that can be formulated as bilevel optimization models, where decisions are organized in a hierarchical structure. This paradigm…
AdaGrad-Diff: A New Version of the Adaptive Gradient Algorithm
Matia Bojovic, Saverio Salzo, Massimiliano Pontil
Vanilla gradient methods are often highly sensitive to the choice of stepsize, which typically requires manual tuning. Adaptive methods alleviate this issue and have therefore beco…
Hyperparameter Optimization in Machine Learning
Luca Franceschi, Michele Donini, Valerio Perrone +5
Hyperparameters are configuration variables controlling the behavior of machine learning algorithms. They are ubiquitous in machine learning and artificial intelligence and the cho…
A conversion theorem and minimax optimality for continuum contextual bandits
Arya Akhavan, Karim Lounici, Massimiliano Pontil +1
We study the contextual continuum bandits problem, where the learner sequentially receives a side information vector and has to choose an action in a convex set, minimizing a funct…
DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products
Julien Siems, Timur Carstensen, Arber Zela +3
Linear Recurrent Neural Networks (linear RNNs) have emerged as competitive alternatives to Transformers for sequence modeling, offering efficient training and linear-time inference…