1.9k citations · 6.5k across the 132 of their papers we have counts for
15 papers · 1 filter
An Accelerated Communication-Efficient Primal-Dual Optimization Framework for Structured Machine Learning
Chenxin Ma, Martin Jaggi, Frank E. Curtis +2
Distributed optimization algorithms are essential for training machine learning models on very large-scale datasets. However, they often suffer from communication bottlenecks. Conf…
The Implicit Bias of Gradient Descent on Separable Data
Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson +2
We examine gradient descent on unregularized logistic regression problems, with homogeneous linear predictors on linearly separable datasets. We show the predictor converges to the…
Stochastic Nonconvex Optimization with Large Minibatches
Weiran Wang, Nathan Srebro
We study stochastic optimization of nonconvex loss functions, which are typical objectives for training neural networks. We propose stochastic approximation algorithms which optimi…
Lower Bound for Randomized First Order Convex Optimization
Blake Woodworth, Nathan Srebro
We provide an explicit construction and direct proof for the lower bound on the number of first order oracle accesses required for a randomized algorithm to minimize a convex Lipsc…
Exploring Generalization in Deep Learning
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester +1
With a goal of understanding what drives generalization in deep networks, we consider several recently suggested explanations, including norm-based control, sharpness and robustnes…
A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
Behnam Neyshabur, Srinadh Bhojanapalli, Nathan Srebro
We present a generalization bound for feedforward neural networks in terms of the product of the spectral norm of the layers and the Frobenius norm of the weights. The generalizati…