167 citations · 280 across the 9 of their papers we have counts for
9 papers
Plex: Towards Reliability using Pretrained Large Model Extensions
Dustin Tran, Jeremiah Liu, Michael W. Dusenberry +23
A recent trend in artificial intelligence is the use of pretrained models for language and vision tasks, which have achieved extraordinary performance but also puzzling failures. P…
Big-Step-Little-Step: Efficient Gradient Methods for Objectives with Multiple Scales
Jonathan Kelner, Annie Marsden, Vatsal Sharan +3
We provide new gradient-based methods for efficiently solving a broad class of ill-conditioned optimization problems. We consider the problem of minimizing a function $f : \mathbb{…
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective
Margalit Glasgow, Honglin Yuan, Tengyu Ma
Federated Averaging (FedAvg), also known as Local SGD, is one of the most popular algorithms in Federated Learning (FL). Despite its simplicity and popularity, the convergence rate…
What Do We Mean by Generalization in Federated Learning?
Honglin Yuan, Warren Morningstar, Lin Ning +1
Federated learning data is drawn from a distribution of distributions: clients are drawn from a meta-distribution, and their data are drawn from local data distributions. Thus gene…
A Field Guide to Federated Optimization
Jianyu Wang, Zachary Charles, Zheng Xu +50
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy prote…
Federated Composite Optimization
Honglin Yuan, Manzil Zaheer, Sashank Reddi
Federated Learning (FL) is a distributed learning paradigm that scales on-device learning collaboratively and privately. Standard FL algorithms such as FedAvg are primarily geared…
SHREC 2020 track: 6D Object Pose Estimation
Honglin Yuan, Remco C. Veltkamp, Georgios Albanis +3
6D pose estimation is crucial for augmented reality, virtual reality, robotic manipulation and visual navigation. However, the problem is challenging due to the variety of objects…
Federated Accelerated Stochastic Gradient Descent
Honglin Yuan, Tengyu Ma
We propose Federated Accelerated Stochastic Gradient Descent (FedAc), a principled acceleration of Federated Averaging (FedAvg, also known as Local SGD) for distributed optimizatio…
Global Optimization with Orthogonality Constraints via Stochastic Diffusion on Manifold
Honglin Yuan, Xiaoyi Gu, Rongjie Lai +1
Orthogonality constrained optimization is widely used in applications from science and engineering. Due to the nonconvex orthogonality constraints, many numerical algorithms often…