8 papers
Convergence of zeroth-order proximal point algorithms in the high-temperature regime
Emanuele Naldi, Hippolyte Labarrière, Cesare Molinari +1
Efficient methods for non-convex black-box optimization largely rely on sampling. In this context, the Zeroth-Order Proximal Operator (ZOPO) and the corresponding Zeroth-Order Prox…
SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching
Hippolyte Labarrière, Cesare Molinari, Silvia Villa +1
Black-Box Variational Inference (BBVI) typically relies on Stochastic Gradient Descent (SGD) to optimize the Evidence Lower Bound (ELBO). However, the stochastic gradients in BBVI…
Learning Multi-Index Models with Hyper-Kernel Ridge Regression
Shuo Huang, Hippolyte Labarrière, Ernesto De Vito +2
Deep neural networks excel in high-dimensional problems, outperforming models such as kernel methods, which suffer from the curse of dimensionality. However, the theoretical founda…
Optimization Insights into Deep Diagonal Linear Networks
Hippolyte Labarrière, Cesare Molinari, Lorenzo Rosasco +2
Gradient-based methods successfully train highly overparameterized models in practice, even though the associated optimization problems are markedly nonconvex. Understanding the me…
Strong Convergence of FISTA Iterates under H{ö}lderian and Quadratic Growth Conditions
Jean-François Aujol, Charles Dossal, Hippolyte Labarrière +1
Introduced by Beck and Teboulle, FISTA (for Fast Iterative Shrinkage-Thresholding Algorithm) is a first-order method widely used in convex optimization. Adapted from Nesterov's acc…
Heavy Ball Momentum for Non-Strongly Convex Optimization
Jean-François Aujol, Charles Dossal, Hippolyte Labarrière +1
When considering the minimization of a quadratic or strongly convex function, it is well known that first-order methods involving an inertial term weighted by a constant-in-time pa…
Parameter-Free FISTA by Adaptive Restart and Backtracking
Jean-François Aujol, Luca Calatroni, Charles Dossal +2
We consider a combined restarting and adaptive backtracking strategy for the popular Fast Iterative Shrinking-Thresholding Algorithm frequently employed for accelerating the conver…
Fast convergence of inertial dynamics with Hessian-driven damping under geometry assumptions
Jean-François Aujol, Charles Dossal, Văn Hào Hoàng +2
First-order optimization algorithms can be considered as a discretization of ordinary differential equations (ODEs) \cite{su2014differential}. In this perspective, studying the pro…