1.9k citations · 6.5k across the 132 of their papers we have counts for
9 papers · 2 filters
Stochastic Nonconvex Optimization with Large Minibatches
Weiran Wang, Nathan Srebro
We study stochastic optimization of nonconvex loss functions, which are typical objectives for training neural networks. We propose stochastic approximation algorithms which optimi…
Exploring Generalization in Deep Learning
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester +1
With a goal of understanding what drives generalization in deep networks, we consider several recently suggested explanations, including norm-based control, sharpness and robustnes…
A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
Behnam Neyshabur, Srinadh Bhojanapalli, Nathan Srebro
We present a generalization bound for feedforward neural networks in terms of the product of the spectral norm of the layers and the Frobenius norm of the weights. The generalizati…
Geometry of Optimization and Implicit Regularization in Deep Learning
Behnam Neyshabur, Ryota Tomioka, Ruslan Salakhutdinov +1
We argue that the optimization plays a crucial role in generalization of deep learning models through implicit regularization. We do this by demonstrating that generalization abili…
Communication-efficient Algorithms for Distributed Stochastic Principal Component Analysis
Dan Garber, Ohad Shamir, Nathan Srebro
We study the fundamental problem of Principal Component Analysis in a statistical distributed setting in which each machine out of stores a sample of points sampled i.i.d.…
Stochastic Approximation for Canonical Correlation Analysis
Raman Arora, Teodor V. Marinov, Poorya Mianjy +1
We propose novel first-order stochastic approximation algorithms for canonical correlation analysis (CCA). Algorithms presented are instances of inexact matrix stochastic gradient…