6.9k citations
- Meta (United Kingdom)GB158 papers
- Meta (Israel)IL62 papers
- Menlo SchoolUS51 papers
- New York UniversityUS49 papers
- Carnegie Mellon UniversityUS33 papers
- University of California, BerkeleyUS31 papers
- The University of Texas at AustinUS28 papers
- Google (United States)US27 papers
- Georgia Institute of TechnologyUS26 papers
- Stanford UniversityUS25 papers
- Cornell UniversityUS24 papers
- University of WashingtonUS20 papers
14 papers · 2 filters
Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations
Yuanzhi Li, Tengyu Ma, Hongyang Zhang
We show that the gradient descent algorithm provides an implicit regularization effect in the learning of over-parameterized matrix factorization models and one-hidden-layer neural…
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima
Simon S. Du, Jason D. Lee, Yuandong Tian +2
We consider the problem of learning a one-hidden-layer neural network with non-overlapping convolutional layer and ReLU activation, i.e., $f(\mathbf{Z}, \mathbf{w}, \mathbf{a}) = \…
ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases
Pierre Stock, Moustapha Cisse
ConvNets and Imagenet have driven the recent success of deep learning for image classification. However, the marked slowdown in performance improvement combined with the lack of ro…
When is a Convolutional Filter Easy To Learn?
Simon S. Du, Jason D. Lee, Yuandong Tian
We analyze the convergence of (stochastic) gradient descent algorithm for learning a convolutional filter with Rectified Linear Unit (ReLU) activation function. Our analysis does n…
A hierarchical loss and its problems when classifying non-hierarchically
Cinna Wu, Mark Tygert, Yann LeCun
Failing to distinguish between a sheepdog and a skyscraper should be worse and penalized more than failing to distinguish between a sheepdog and a poodle; after all, sheepdogs and…
Gradient Episodic Memory for Continual Learning
David Lopez-Paz, Marc'Aurelio Ranzato
One major obstacle towards AI is the poor ability of models to solve new problems quicker, and without forgetting previously acquired knowledge. To better understand this issue, we…