15 citations · 30 across the 16 of their papers we have counts for
9 papers · 1 filter
Convergence of Distributed Adaptive Optimization with Local Updates
Ziheng Cheng, Margalit Glasgow
We study distributed adaptive algorithms with local updates (intermittent communication). Despite the great empirical success of adaptive methods in distributed training of modern…
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
Kumar Kshitij Patel, Margalit Glasgow, Ali Zindari +5
Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD. Despite this success, theoretically…
SGD Finds then Tunes Features in Two-Layer Neural Networks with near-Optimal Sample Complexity: A Case Study in the XOR problem
Margalit Glasgow
In this work, we consider the optimization process of minibatch stochastic gradient descent (SGD) on a 2-layer neural network with data separated by a quadratic ground truth functi…
Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time
Arvind Mahankali, Jeff Z. Haochen, Kefan Dong +2
Despite recent theoretical progress on the non-convex optimization of two-layer neural networks, it is still an open question whether gradient descent on neural networks without un…
Tight Bounds for -Regret via the Decision-Estimation Coefficient
Margalit Glasgow, Alexander Rakhlin
In this work, we give a statistical characterization of the -regret for arbitrary structured bandit problems, the regret which arises when comparing against a benchmark that is…
Feature Dropout: Revisiting the Role of Augmentations in Contrastive Learning
Alex Tamkin, Margalit Glasgow, Xiluo He +1
What role do augmentations play in contrastive learning? Recent work suggests that good augmentations are label-preserving with respect to a specific downstream task. We complicate…
Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence
Margalit Glasgow, Colin Wei, Mary Wootters +1
A major challenge in modern machine learning is theoretically understanding the generalization properties of overparameterized models. Many existing tools rely on uniform convergen…
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective
Margalit Glasgow, Honglin Yuan, Tengyu Ma
Federated Averaging (FedAvg), also known as Local SGD, is one of the most popular algorithms in Federated Learning (FL). Despite its simplicity and popularity, the convergence rate…
Asynchronous Distributed Optimization with Stochastic Delays
Margalit Glasgow, Mary Wootters
We study asynchronous finite sum minimization in a distributed-data setting with a central parameter server. While asynchrony is well understood in parallel settings where the data…