8 papers
Optimization by Reparametrization: Time-Warped Mirror Flows with Singular Geometry
Cristian Vega, Cesare Molinari, Lorenzo Rosasco +1
Solving data-driven problems requires defining complex models and fitting them to data, neural networks being a motivating example. The fitting procedure can be seen as an optimiza…
Langevin for Nonconvex Optimization: Exact, Inexact and Zeroth-Order
Emanuele Naldi, Marco Rando, Lorenzo Rosasco +1
We study Langevin-based methods for non-convex optimization under smoothness and dissipativity assumptions, focusing on non-asymptotic expected excess-risk bounds rather than sampl…
Convergence of zeroth-order proximal point algorithms in the high-temperature regime
Emanuele Naldi, Hippolyte Labarrière, Cesare Molinari +1
Efficient methods for non-convex black-box optimization largely rely on sampling. In this context, the Zeroth-Order Proximal Operator (ZOPO) and the corresponding Zeroth-Order Prox…
SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching
Hippolyte Labarrière, Cesare Molinari, Silvia Villa +1
Black-Box Variational Inference (BBVI) typically relies on Stochastic Gradient Descent (SGD) to optimize the Evidence Lower Bound (ELBO). However, the stochastic gradients in BBVI…
A Proximal Stochastic Variance Reduced Zeroth-order Algorithm using Structured Finite-Differences
Marco Rando, Cheik Traoré, Cesare Molinari +2
Minimizing finite sums of functions is a central problem in optimization, arising in numerous practical applications. Such problems are commonly addressed using first-order optimiz…
Preconditioned primal-dual dynamics in convex optimization: non-ergodic convergence rates
Vassilis Apidopoulos, Cesare Molinari, Juan Peypouquet +1
We introduce and analyze a continuous primal-dual dynamical system in the context of the minimization problem , where and are convex functions and is a line…
A Structured Tour of Optimization with Finite Differences
Marco Rando, Cesare Molinari, Lorenzo Rosasco +1
Finite-difference methods are widely used for zeroth-order optimization in settings where gradient information is unavailable or expensive to compute. These procedures mimic first-…
Model Consistency of the Iterative Regularization of Dual Ascent for Low-Complexity Regularization
Jie Gao, Cesare Molinari, Silvia Villa +1
Regularization is a core component of modern inverse problems, as it helps establish the well-posedness of the solution of interest. Popular regularization approaches include varia…