1.6k citations · 3.6k across the 48 of their papers we have counts for
4 papers · 1 filter
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri +3
Federated Averaging (FedAvg) has emerged as the algorithm of choice for federated learning due to its simplicity and low communication cost. However, in spite of recent research ef…
On the Convergence of Adam and Beyond
Sashank J. Reddi, Satyen Kale, Sanjiv Kumar
Several recently proposed stochastic optimization methods that have been successfully used in training deep networks such as RMSProp, Adam, Adadelta, Nadam are based on using gradi…
Hypothesis Set Stability and Generalization
Dylan J. Foster, Spencer Greenberg, Satyen Kale +3
We present a study of generalization for data-dependent hypothesis sets. We give a general learning guarantee for data-dependent hypothesis sets based on a notion of transductive R…
Escaping Saddle Points with Adaptive Gradient Methods
Matthew Staib, Sashank J. Reddi, Satyen Kale +2
Adaptive methods such as Adam and RMSProp are widely used in deep learning but are not well understood. In this paper, we seek a crisp, clean and precise characterization of their…