536 citations · 4.4k across the 97 of their papers we have counts for
5 papers · 1 filter
Global Non-convex Optimization with Discretized Diffusions
Murat A. Erdogdu, Lester Mackey, Ohad Shamir
An Euler discretization of the Langevin diffusion is known to converge to the global minimizers of certain convex and non-convex optimization problems. We show that this property h…
Exponential Convergence Time of Gradient Descent for One-Dimensional Deep Linear Neural Networks
Ohad Shamir
We study the dynamics of gradient descent on objective functions of the form (with respect to scalar parameters ), which arise in the conte…
A Tight Convergence Analysis for Stochastic Gradient Descent with Delayed Updates
Yossi Arjevani, Ohad Shamir, Nathan Srebro
We provide tight finite-time convergence bounds for gradient descent and stochastic gradient descent on quadratic functions, when the gradients are delayed and reflect iterates fro…
Are ResNets Provably Better than Linear Predictors?
Ohad Shamir
A residual network (or ResNet) is a standard deep neural net architecture, with state-of-the-art performance across numerous applications. The main premise of ResNets is that they…
Detecting Correlations with Little Memory and Communication
Yuval Dagan, Ohad Shamir
We study the problem of identifying correlations in multivariate data, under information constraints: Either on the amount of memory that can be used by the algorithm, or the amoun…