1.6k citations · 3.6k across the 48 of their papers we have counts for
9 papers · 1 filter
From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descent
Satyen Kale, Jason D. Lee, Chris De Sa +2
Stochastic Gradient Descent (SGD) has been the method of choice for learning large-scale non-convex models. While a general analysis of when SGD works has been elusive, there has b…
Private Matrix Approximation and Geometry of Unitary Orbits
Oren Mangoubi, Yikai Wu, Satyen Kale +2
Consider the following optimization problem: Given matrices and , maximize where varies over the unitary group . Th…
Self-Consistency of the Fokker-Planck Equation
Zebang Shen, Zhenfu Wang, Satyen Kale +3
The Fokker-Planck equation (FPE) is the partial differential equation that governs the density evolution of the Itô process and is of great importance to the literature of statisti…
Mixed Federated Learning: Joint Decentralized and Centralized Learning
Sean Augenstein, Andrew Hard, Lin Ning +4
Federated learning (FL) enables learning from decentralized privacy-sensitive data, with computations on raw data confined to take place at edge clients. This paper introduces mixe…
On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data
Jianyu Wang, Rudrajit Das, Gauri Joshi +3
Existing theory predicts that data heterogeneity will degrade the performance of the Federated Averaging (FedAvg) algorithm in federated learning. However, in practice, the simple…
Beyond Uniform Lipschitz Condition in Differentially Private Optimization
Rudrajit Das, Satyen Kale, Zheng Xu +2
Most prior results on differentially private stochastic gradient descent (DP-SGD) are derived under the simplistic assumption of uniform Lipschitzness, i.e., the per-sample gradien…