105 citations · 160 across the 25 of their papers we have counts for
29 papers
A Bregman Proximal Viewpoint on Neural Operators
Abdel-Rahim Mezidi, Jordan Patracone, Saverio Salzo +4
We present several advances on neural operators by viewing the action of operator layers as the minimizers of Bregman regularized optimization problems over Banach function spaces.…
AdaGrad does not adapt to Hölder-smoothness for composite objectives
Matia Bojovic, Saverio Salzo, Massimiliano Pontil
We exhibit a simple deterministic one-dimensional convex composite optimization problem for which AdaGrad scheme does not achieve the classical convergence rate $\mathcal{O}(n^{-(1…
Bilevel learning
Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo +1
Bilevel learning refers to machine learning problems that can be formulated as bilevel optimization models, where decisions are organized in a hierarchical structure. This paradigm…
Iteration Complexity of Frank-Wolfe and Its Variants for Bilevel Optimization
Anthony Palmieri, Francesco Rinaldi, Saverio Salzo +1
We study Frank-Wolfe (FW) methods for constrained bilevel optimization when the lower-level problem is solved only approximately, yielding biased and inexact hypergradients. We ana…
AdaGrad-Diff: A New Version of the Adaptive Gradient Algorithm
Matia Bojovic, Saverio Salzo, Massimiliano Pontil
Vanilla gradient methods are often highly sensitive to the choice of stepsize, which typically requires manual tuning. Adaptive methods alleviate this issue and have therefore beco…
The iterates of FISTA converge even under inexact computations and stochastic gradients
Saverio Salzo
Very recently, the papers "Point Convergence of Nesterov's Accelerated Gradient Method: An AI-Assisted Proof" by Jang and Ryu, and "The Iterates of Nesterov's Accelerated Algorithm…
An Improved Analysis of the Clipped Stochastic subGradient Method under Heavy-Tailed Noise
Daniela Angela Parletta, Andrea Paudice, Saverio Salzo
In this paper, we provide novel optimal (or near optimal) convergence rates for a clipped version of the stochastic subgradient method. We consider nonsmooth convex problems over p…
Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates
Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo
We study the problem of efficiently computing the derivative of the fixed-point of a parametric nondifferentiable contraction map. This problem has wide applications in machine lea…
Relax and penalize: a new bilevel approach to mixed-binary hyperparameter optimization
Sara Venturini, Marianna de Santis, Jordan Patracone +3
In recent years, bilevel approaches have become very popular to efficiently estimate high-dimensional hyperparameters of machine learning models. However, to date, binary parameter…
Variance reduction techniques for stochastic proximal point algorithms
Cheik Traoré, Vassilis Apidopoulos, Saverio Salzo +1
In the context of finite sums minimization, variance reduction techniques are widely used to improve the performance of state-of-the-art stochastic gradient methods. Their practica…
High Probability Bounds for Stochastic Subgradient Schemes with Heavy Tailed Noise
Daniela A. Parletta, Andrea Paudice, Massimiliano Pontil +1
In this work we study high probability bounds for stochastic subgradient methods under heavy tailed noise. In this setting the noise is only assumed to have finite variance as oppo…
Bilevel Optimization with a Lower-level Contraction: Optimal Sample Complexity without Warm-start
Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo
We analyse a general class of bilevel problems, in which the upper-level problem consists in the minimization of a smooth objective function and the lower-level problem is to find…
Convergence of an Asynchronous Block-Coordinate Forward-Backward Algorithm for Convex Composite Optimization
Cheik Traoré, Saverio Salzo, Silvia Villa
In this paper, we study the convergence properties of a randomized block-coordinate descent algorithm for the minimization of a composite convex objective function, where the block…
Convergence of Batch Greenkhorn for Regularized Multimarginal Optimal Transport
Vladimir Kostic, Saverio Salzo, Massimilano Pontil
In this work we propose a batch version of the Greenkhorn algorithm for multimarginal regularized optimal transport problems. Our framework is general enough to cover, as particula…
The method of Bregman projections in deterministic and stochastic convex feasibility problems
Vladimir Kostic, Saverio Salzo
In this work we study the method of Bregman projections for deterministic and stochastic convex feasibility problems with three types of control sequences for the selection of sets…
Convergence Properties of Stochastic Hypergradients
Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo
Bilevel optimization problems are receiving increasing attention in machine learning as they provide a natural framework for hyperparameter optimization and meta-learning. A key st…
On the Iteration Complexity of Hypergradient Computation
Riccardo Grazzi, Luca Franceschi, Massimiliano Pontil +1
We study a general class of bilevel problems, consisting in the minimization of an upper-level objective which depends on the solution to a parametric fixed-point equation. Importa…
Efficient Tensor Kernel methods for sparse regression
Feliks Hibraj, Marcello Pelillo, Saverio Salzo +1
Recently, classical kernel methods have been extended by the introduction of suitable tensor kernels so to promote sparsity in the solution of the underlying regression problem. In…
Parallel Random Block-Coordinate Forward-Backward Algorithm: A Unified Convergence Analysis
Saverio Salzo, Silvia Villa
We study the block-coordinate forward-backward algorithm in which the blocks are updated in a random and possibly parallel manner, according to arbitrary probabilities. The algorit…
Sinkhorn Barycenters with Free Support via Frank-Wolfe Algorithm
Giulia Luise, Saverio Salzo, Massimiliano Pontil +1
We present a novel algorithm to estimate the barycenter of arbitrary probability distributions with respect to the Sinkhorn divergence. Based on a Frank-Wolfe optimization strategy…
Bilevel Programming for Hyperparameter Optimization and Meta-Learning
Luca Franceschi, Paolo Frasconi, Saverio Salzo +2
We introduce a framework based on bilevel programming that unifies gradient-based hyperparameter optimization and meta-learning. We show that an approximate version of the bilevel…
Far-HO: A Bilevel Programming Package for Hyperparameter Optimization and Meta-Learning
Luca Franceschi, Riccardo Grazzi, Massimiliano Pontil +2
In (Franceschi et al., 2018) we proposed a unified mathematical framework, grounded on bilevel programming, that encompasses gradient-based hyperparameter optimization and meta-lea…
Latent Variable Time-varying Network Inference
Federico Tomasi, Veronica Tozzo, Saverio Salzo +1
In many applications of finance, biology and sociology, complex systems involve entities interacting with each other. These processes have the peculiarity of evolving over time and…
Solving -norm regularization with tensor kernels
Saverio Salzo, Johan A. K. Suykens, Lorenzo Rosasco
In this paper, we discuss how a suitable family of tensor kernels can be used to efficiently solve nonparametric extensions of regularized learning methods. Our main contr…