5 papers
Convergence of zeroth-order proximal point algorithms in the high-temperature regime
Emanuele Naldi, Hippolyte Labarrière, Cesare Molinari +1
Efficient methods for non-convex black-box optimization largely rely on sampling. In this context, the Zeroth-Order Proximal Operator (ZOPO) and the corresponding Zeroth-Order Prox…
SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching
Hippolyte Labarrière, Cesare Molinari, Silvia Villa +1
Black-Box Variational Inference (BBVI) typically relies on Stochastic Gradient Descent (SGD) to optimize the Evidence Lower Bound (ELBO). However, the stochastic gradients in BBVI…
Optimization Insights into Deep Diagonal Linear Networks
Hippolyte Labarrière, Cesare Molinari, Lorenzo Rosasco +2
Gradient-based methods successfully train highly overparameterized models in practice, even though the associated optimization problems are markedly nonconvex. Understanding the me…
Learning Multi-Index Models with Hyper-Kernel Ridge Regression
Shuo Huang, Hippolyte Labarrière, Ernesto De Vito +2
Deep neural networks excel in high-dimensional problems, outperforming models such as kernel methods, which suffer from the curse of dimensionality. However, the theoretical founda…
Heavy Ball Momentum for Non-Strongly Convex Optimization
Jean-François Aujol, Charles Dossal, Hippolyte Labarrière +1
When considering the minimization of a quadratic or strongly convex function, it is well known that first-order methods involving an inertial term weighted by a constant-in-time pa…