167 citations · 679 across the 25 of their papers we have counts for
7 papers · 1 filter
Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays
Konstantin Mishchenko, Francis Bach, Mathieu Even +1
The existing analysis of asynchronous stochastic gradient descent (SGD) degrades dramatically when any delay is large, giving the impression that performance depends primarily on t…
A Stochastic Newton Algorithm for Distributed Convex Optimization
Brian Bullins, Kumar Kshitij Patel, Ohad Shamir +2
We propose and analyze a stochastic Newton algorithm for homogeneous distributed stochastic convex optimization, where each machine can calculate stochastic gradients of the same p…
The Minimax Complexity of Distributed Optimization
Blake Woodworth
In this thesis, I study the minimax oracle complexity of distributed stochastic optimization. First, I present the "graph oracle model", an extension of the classic oracle complexi…
Lower Bounds for Non-Convex Stochastic Optimization
Yossi Arjevani, Yair Carmon, John C. Duchi +3
We lower bound the complexity of finding -stationary points (with gradient norm at most ) using stochastic first-order methods. In a well-studied model where algorithms acces…
Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimization
Blake Woodworth, Jialei Wang, Adam Smith +2
We suggest a general oracle-based framework that captures different parallel stochastic optimization settings described by a dependency graph, and derive generic lower bounds in te…
Lower Bound for Randomized First Order Convex Optimization
Blake Woodworth, Nathan Srebro
We provide an explicit construction and direct proof for the lower bound on the number of first order oracle accesses required for a randomized algorithm to minimize a convex Lipsc…
Tight Complexity Bounds for Optimizing Composite Objectives
Blake Woodworth, Nathan Srebro
We provide tight upper and lower bounds on the complexity of minimizing the average of convex functions using gradient and prox oracles of the component functions. We show a si…