4 citations · 9 across the 25 of their papers we have counts for
11 papers · 1 filter
TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuning
Jichao Jiang, Cristian McGee, El Houcine Bergou +3
Full-parameter fine-tuning of large language models (LLMs) incurs substantial optimizer state memory overhead, limiting the model sizes that fit on modern GPUs. Existing approaches…
Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuning
Cristian McGee, El Houcine Bergou, Aritra Dutta
Step-size selection remains a central challenge in large-scale neural network optimization; conservative steps slow convergence, while aggressive steps can destabilize it. We combi…
Beyond LoRA: Is Sparsity-Induced Adaptation Better?
Elijah Cadenhead, Cristian McGee, Xin Li +2
Low-rank adaptation (LoRA) and its variants provide a memory- and compute-efficient alternative to full fine-tuning of pre-trained models. However, questions remain about the compa…
FairEnergy: Contribution-Based Fairness meets Energy Efficiency in Federated Learning
Ouiame Marnissi, Hajar EL Hammouti, El Houcine Bergou
Federated learning (FL) enables collaborative model training across distributed devices while preserving data privacy. However, balancing energy efficiency and fair participation w…
Stabilizing Policy Gradient Methods via Reward Profiling
Shihab Ahmed, El Houcine Bergou, Aritra Dutta +1
Policy gradient methods, which have been extensively studied in the last decade, offer an effective and efficient framework for reinforcement learning problems. However, their perf…
If You Want to Be Robust, Be Wary of Initialization
Sofiane Ennadir, Johannes F. Lutzeyer, Michalis Vazirgiannis +1
Graph Neural Networks (GNNs) have demonstrated remarkable performance across a spectrum of graph-related tasks, however concerns persist regarding their vulnerability to adversaria…
Joint Probability Selection and Power Allocation for Federated Learning
Ouiame Marnissi, Hajar EL Hammouti, El Houcine Bergou
In this paper, we study the performance of federated learning over wireless networks, where devices with a limited energy budget train a machine learning model. The federated learn…
Demystifying the Myths and Legends of Nonconvex Convergence of SGD
Aritra Dutta, El Houcine Bergou, Soumia Boucherouite +3
Stochastic gradient descent (SGD) and its variants are the main workhorses for solving large-scale optimization problems with nonconvex objective functions. Although the convergenc…
Linear Scalarization for Byzantine-robust learning on non-IID data
Latifa Errami, El Houcine Bergou
In this work we study the problem of Byzantine-robust learning when data among clients is heterogeneous. We focus on poisoning attacks targeting the convergence of SGD. Although th…
Personalized Federated Learning with Communication Compression
El Houcine Bergou, Konstantin Burlachenko, Aritra Dutta +1
In contrast to training traditional machine learning (ML) models in data centers, federated learning (FL) trains ML models over local datasets contained on resource-constrained het…
Client Selection in Federated Learning based on Gradients Importance
Ouiame Marnissi, Hajar El Hammouti, El Houcine Bergou
Federated learning (FL) enables multiple devices to collaboratively learn a global model without sharing their personal data. In real-world applications, the different parties are…