3 papers
cs.LG2025
-LoRA: Effective Fine-Tuning via Base Model Rescaling
Aymane El Firdoussi, El Mahdi Chayti, Mohamed El Amine Seddik +1
Fine-tuning has proven to be highly effective in adapting pre-trained models to perform better on new desired tasks with minimal data samples. Among the most widely used approaches…
cs.LG2024
Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory
Aymane El Firdoussi, Mohamed El Amine Seddik, Soufiane Hayou +3
Synthetic data has gained attention for training large language models, but poor-quality data can harm performance (see, e.g., Shumailov et al. (2023); Seddik et al. (2024)). A pot…
cs.LG2024
The Privacy Power of Correlated Noise in Decentralized Learning
Youssef Allouah, Anastasia Koloskova, Aymane El Firdoussi +2
Decentralized learning is appealing as it enables the scalable usage of large amounts of distributed data and resources (without resorting to any central entity), while promoting p…