152 citations · 152 across the 6 of their papers we have counts for
4 papers · 1 filter
Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping
Hsun-Yu Kuo, El Mahdi Chayti, Patrik Reizinger +2
Looped Transformers, which repeatedly apply a shared transformer block, are an architecturally natural fit for variable-length algorithmic tasks. Although they can exhibit strong l…
-LoRA: Effective Fine-Tuning via Base Model Rescaling
Aymane El Firdoussi, El Mahdi Chayti, Mohamed El Amine Seddik +1
Fine-tuning has proven to be highly effective in adapting pre-trained models to perform better on new desired tasks with minimal data samples. Among the most widely used approaches…
Stochastic Difference-of-Convex Optimization with Momentum
El Mahdi Chayti, Martin Jaggi
We study the online stochastic difference-of-convex (DC) program , which covers sparse learning with nonconvex penalties, rob…
Block-Coordinate Frank-Wolfe Optimization for Structural SVMs
Simon Lacoste-Julien, Martin Jaggi, Mark Schmidt +1
We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints. Despite its lower iteration cost, we…