152 citations · 152 across the 6 of their papers we have counts for
6 papers
Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping
Hsun-Yu Kuo, El Mahdi Chayti, Patrik Reizinger +2
Looped Transformers, which repeatedly apply a shared transformer block, are an architecturally natural fit for variable-length algorithmic tasks. Although they can exhibit strong l…
-LoRA: Effective Fine-Tuning via Base Model Rescaling
Aymane El Firdoussi, El Mahdi Chayti, Mohamed El Amine Seddik +1
Fine-tuning has proven to be highly effective in adapting pre-trained models to perform better on new desired tasks with minimal data samples. Among the most widely used approaches…
Stochastic Difference-of-Convex Optimization with Momentum
El Mahdi Chayti, Martin Jaggi
We study the online stochastic difference-of-convex (DC) program , which covers sparse learning with nonconvex penalties, rob…
Second-Order Optimization with Asynchronous Curvature
El Mahdi Chayti, Martin Jaggi
Second-order methods make fast progress per iteration, but forming and factorizing the Hessian often dominates their wall-clock time. We study \emph{Split-Client}, a task-parallel…
Stochastic Optimization with Random Search
El Mahdi Chayti, Taha El Bakkali El Kadi, Omar Saadi +1
We revisit random search for stochastic optimization, where only noisy function evaluations are available. We show that the method works under weaker smoothness assumptions than pr…
Block-Coordinate Frank-Wolfe Optimization for Structural SVMs
Simon Lacoste-Julien, Martin Jaggi, Mark Schmidt +1
We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints. Despite its lower iteration cost, we…