2 citations · 5 across the 16 of their papers we have counts for
10 papers · 1 filter
Stochastic Zeroth-Order Optimization Under Heavy-Tailed Noise
Taha El Bakkali, El Mahdi Chayti, Qiuyi Zhang +2
We study stochastic zeroth-order (ZO) optimization of smooth nonconvex objectives under heavy-tailed sample-gradient noise. This regime is motivated by empirical evidence that grad…
Stochastic Compositional Optimization via Hybrid Momentum Frank--Wolfe
El Mahdi Chayti, Mohamad Fakhouri
Stochastic compositional optimization minimizes over a compact convex set when the inner map is available only through noisy samples. We propose Hybrid Momentum St…
Nonsmooth Nonconvex Optimization from Noisy Comparisons: Unbiased Gap Estimation and the Price of Preference Feedback
Taha El Bakkali, El Mahdi Chayti, Omar Saadi
We study the minimization of -Lipschitz, possibly nonsmooth and nonconvex from noisy pairwise comparisons: comparing and returns a bit equal to with pro…
RanSOM: Second-Order Momentum with Randomized Scaling for Constrained and Unconstrained Optimization
El Mahdi Chayti, Mohamad Fakhouri
Second-order momentum corrects the lag of momentum behind the current gradient with a Hessian-vector product (HVP), but existing corrections need either a Lipschitz Hessian or extr…
Second-Order Optimization with Asynchronous Curvature
El Mahdi Chayti, Martin Jaggi
Second-order methods make fast progress per iteration, but forming and factorizing the Hessian often dominates their wall-clock time. We study \emph{Split-Client}, a task-parallel…
Stochastic Optimization with Random Search
El Mahdi Chayti, Taha El Bakkali El Kadi, Omar Saadi +1
We revisit random search for stochastic optimization, where only noisy function evaluations are available. We show that the method works under weaker smoothness assumptions than pr…
Faster Gradient Methods for Highly-Smooth Stochastic Bilevel Optimization
Lesi Chen, Junru Li, El Mahdi Chayti +1
This paper studies the complexity of finding an -stationary point for stochastic bilevel optimization when the upper-level problem is nonconvex and the lower-level problem is st…
Improving Stochastic Cubic Newton with Momentum
El Mahdi Chayti, Nikita Doikov, Martin Jaggi
We study stochastic second-order methods for solving general non-convex optimization problems. We propose using a special version of momentum to stabilize the stochastic gradient a…
Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods
El Mahdi Chayti, Nikita Doikov, Martin Jaggi
We study stochastic Cubic Newton methods for solving general possibly non-convex minimization problems. We propose a new framework, which we call the helper framework, that provide…
Second-order optimization with lazy Hessians
Nikita Doikov, El Mahdi Chayti, Martin Jaggi
We analyze Newton's method with lazy Hessian updates for solving general possibly non-convex optimization problems. We propose to reuse a previously seen Hessian for several iterat…