167 citations · 679 across the 25 of their papers we have counts for
6 papers · 1 filter
A Stochastic Newton Algorithm for Distributed Convex Optimization
Brian Bullins, Kumar Kshitij Patel, Ohad Shamir +2
We propose and analyze a stochastic Newton algorithm for homogeneous distributed stochastic convex optimization, where each machine can calculate stochastic gradients of the same p…
The Minimax Complexity of Distributed Optimization
Blake Woodworth
In this thesis, I study the minimax oracle complexity of distributed stochastic optimization. First, I present the "graph oracle model", an extension of the classic oracle complexi…
A Field Guide to Federated Optimization
Jianyu Wang, Zachary Charles, Zheng Xu +50
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy prote…
An Even More Optimal Stochastic Optimization Algorithm: Minibatching and Interpolation Learning
Blake Woodworth, Nathan Srebro
We present and analyze an algorithm for optimizing smooth and convex or strongly convex objectives using minibatch stochastic gradient estimates. The algorithm is optimal with resp…
On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent
Shahar Azulay, Edward Moroshko, Mor Shpigel Nacson +4
Recent work has highlighted the role of initialization scale in determining the structure of the solutions that gradient methods converge to. In particular, it was shown that large…
The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication
Blake Woodworth, Brian Bullins, Ohad Shamir +1
We resolve the min-max complexity of distributed stochastic convex optimization (up to a log factor) in the intermittent communication setting, where machines work in parallel…