19 citations · 38 across the 14 of their papers we have counts for
8 papers · 1 filter
Beyond LoRA: Is Sparsity-Induced Adaptation Better?
Elijah Cadenhead, Cristian McGee, Xin Li +2
Low-rank adaptation (LoRA) and its variants provide a memory- and compute-efficient alternative to full fine-tuning of pre-trained models. However, questions remain about the compa…
MFedAQI: Multimodal Federated Learning for Air Quality Prediction on Heterogeneous Edge Devices
Manjil Nepal, Kimsie Phan, Tamoghna Ojha +2
Accurate air quality prediction is essential for public health, environmental monitoring, and industrial safety. However, most existing approaches rely on centralized learning para…
Stabilizing Policy Gradient Methods via Reward Profiling
Shihab Ahmed, El Houcine Bergou, Aritra Dutta +1
Policy gradient methods, which have been extensively studied in the last decade, offer an effective and efficient framework for reinforcement learning problems. However, their perf…
Kolmogorov-Arnold Attention: Is Learnable Attention Better For Vision Transformers?
Subhajit Maity, Killian Hitsman, Xin Li +1
Kolmogorov-Arnold networks (KANs) are a remarkable innovation that consists of learnable activation functions, with the potential to capture more complex relationships from data. P…
Demystifying the Myths and Legends of Nonconvex Convergence of SGD
Aritra Dutta, El Houcine Bergou, Soumia Boucherouite +3
Stochastic gradient descent (SGD) and its variants are the main workhorses for solving large-scale optimization problems with nonconvex objective functions. Although the convergenc…
Personalized Federated Learning with Communication Compression
El Houcine Bergou, Konstantin Burlachenko, Aritra Dutta +1
In contrast to training traditional machine learning (ML) models in data centers, federated learning (FL) trains ML models over local datasets contained on resource-constrained het…