152 citations · 152 across the 6 of their papers we have counts for
4 papers · 1 filter
-LoRA: Effective Fine-Tuning via Base Model Rescaling
Aymane El Firdoussi, El Mahdi Chayti, Mohamed El Amine Seddik +1
Fine-tuning has proven to be highly effective in adapting pre-trained models to perform better on new desired tasks with minimal data samples. Among the most widely used approaches…
Stochastic Difference-of-Convex Optimization with Momentum
El Mahdi Chayti, Martin Jaggi
We study the online stochastic difference-of-convex (DC) program , which covers sparse learning with nonconvex penalties, rob…
Second-Order Optimization with Asynchronous Curvature
El Mahdi Chayti, Martin Jaggi
Second-order methods make fast progress per iteration, but forming and factorizing the Hessian often dominates their wall-clock time. We study \emph{Split-Client}, a task-parallel…
Stochastic Optimization with Random Search
El Mahdi Chayti, Taha El Bakkali El Kadi, Omar Saadi +1
We revisit random search for stochastic optimization, where only noisy function evaluations are available. We show that the method works under weaker smoothness assumptions than pr…