1.6k citations · 3.9k across the 52 of their papers we have counts for
8 papers · 1 filter
Federated Composite Optimization
Honglin Yuan, Manzil Zaheer, Sashank Reddi
Federated Learning (FL) is a distributed learning paradigm that scales on-device learning collaboratively and privately. Standard FL algorithms such as FedAvg are primarily geared…
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning
Sai Praneeth Karimireddy, Martin Jaggi, Satyen Kale +4
Federated learning (FL) is a challenging setting for optimization due to the heterogeneity of the data across different clients which gives rise to the client drift phenomenon. In…
Connections are Expressive Enough: Universal Approximability of Sparse Transformers
Chulhee Yun, Yin-Wen Chang, Srinadh Bhojanapalli +3
Recently, Transformer networks have redefined the state of the art in many NLP tasks. However, these models suffer from quadratic computational cost in the input sequence length $n…
Why distillation helps: a statistical perspective
Aditya Krishna Menon, Ankit Singh Rawat, Sashank J. Reddi +2
Knowledge distillation is a technique for improving the performance of a simple "student" model by replacing its one-hot training labels with a distribution over labels obtained fr…
Doubly-stochastic mining for heterogeneous retrieval
Ankit Singh Rawat, Aditya Krishna Menon, Andreas Veit +3
Modern retrieval problems are characterised by training sets with potentially billions of labels, and heterogeneous data distributions across subpopulations (e.g., users of a retri…
Adaptive Federated Optimization
Sashank Reddi, Zachary Charles, Manzil Zaheer +5
Federated learning is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own trainin…