12 citations · 50 across the 49 of their papers we have counts for
8 papers · 2 filters
Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Masks
MohammadReza Davari, Eugene Belilovsky
The rapid development of AI systems has been greatly influenced by the emergence of foundation models. A common approach for targeted problems involves fine-tuning these pre-traine…
Meta-learning Optimizers for Communication-Efficient Learning
Charles-Étienne Joseph, Benjamin Thérien, Abhinav Moudgil +2
Communication-efficient variants of SGD, specifically local SGD, have received a great deal of interest in recent years. These approaches compute multiple gradient steps locally on…
Adversarial Attacks on the Interpretation of Neuron Activation Maximization
Geraldin Nanfack, Alexander Fulleringer, Jonathan Marty +2
The internal functional behavior of trained Deep Neural Networks is notoriously difficult to interpret. Activation-maximization approaches are one set of techniques used to interpr…
Can Forward Gradient Match Backpropagation?
Louis Fournier, Stéphane Rivaud, Eugene Belilovsky +2
Forward Gradients - the idea of using directional derivatives in forward differentiation mode - have recently been shown to be utilizable for neural network training while avoiding…
: Accelerating Asynchronous Communication in Decentralized Deep Learning
Adel Nabli, Eugene Belilovsky, Edouard Oyallon
Distributed training of Deep Learning models has been critical to many recent successes in the field. Current standard methods primarily rely on synchronous centralized algorithms…
Guiding The Last Layer in Federated Learning with Pre-Trained Models
Gwen Legate, Nicolas Bernier, Lucas Caccia +2
Federated Learning (FL) is an emerging paradigm that allows a model to be trained across a number of participants without sharing data. Recent works have begun to consider the effe…
Re-Weighted Softmax Cross-Entropy to Control Forgetting in Federated Learning
Gwen Legate, Lucas Caccia, Eugene Belilovsky
In Federated Learning, a global model is learned by aggregating model updates computed at a set of independent client nodes, to reduce communication costs multiple gradient steps a…
Local Learning with Neuron Groups
Adeetya Patel, Michael Eickenberg, Eugene Belilovsky
Traditional deep network training methods optimize a monolithic objective function jointly for all the components. This can lead to various inefficiencies in terms of potential par…