117 citations · 411 across the 25 of their papers we have counts for
7 papers · 1 filter
Resource-Efficient Federated Learning
Ahmed M. Abdelmoniem, Atal Narayan Sahu, Marco Canini +1
Federated Learning (FL) enables distributed training by learners using local data, thereby enhancing privacy and reducing communication. However, it presents numerous challenges re…
Rethinking gradient sparsification as total error minimization
Atal Narayan Sahu, Aritra Dutta, Ahmed M. Abdelmoniem +3
Gradient compression is a widely-established remedy to tackle the communication bottleneck in distributed training of large deep neural networks (DNNs). Under the error-feedback fr…
With Great Freedom Comes Great Opportunity: Rethinking Resource Allocation for Serverless Functions
Muhammad Bilal, Marco Canini, Rodrigo Fonseca +1
Current serverless offerings give users a limited degree of flexibility for configuring the resources allocated to their function invocations by either coupling memory and CPU reso…
AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly
Yuchen Jin, Tianyi Zhou, Liangyu Zhao +4
The learning rate (LR) schedule is one of the most important hyper-parameters needing careful tuning in training DNNs. However, it is also one of the least automated parts of machi…
RDMA is Turing complete, we just did not know it yet!
Waleed Reda, Marco Canini, Dejan Kostić +1
It is becoming increasingly popular for distributed systems to exploit offload to reduce load on the CPU. Remote Direct Memory Access (RDMA) offload, in particular, has become popu…
On the Impact of Device and Behavioral Heterogeneity in Federated Learning
Ahmed M. Abdelmoniem, Chen-Yu Ho, Pantelis Papageorgiou +2
Federated learning (FL) is becoming a popular paradigm for collaborative learning over distributed, private datasets owned by non-trusting entities. FL has seen successful deployme…
An Efficient Statistical-based Gradient Compression Technique for Distributed Training Systems
Ahmed M. Abdelmoniem, Ahmed Elzanaty, Mohamed-Slim Alouini +1
The recent many-fold increase in the size of deep neural networks makes efficient distributed training challenging. Many proposals exploit the compressibility of the gradients and…