1.6k citations · 11.3k across the 73 of their papers we have counts for
6 papers · 1 filter
Identity Crisis: Memorization and Generalization under Extreme Overparameterization
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
We study the interplay between memorization and generalization of overparameterized networks in the extreme case of a single training example and an identity-mapping task. We exami…
Are All Layers Created Equal?
Chiyuan Zhang, Samy Bengio, Yoram Singer
Understanding deep neural networks is a major research objective with notable experimental and theoretical attention in recent years. The practical success of excessively large net…
Predicting the Generalization Gap in Deep Networks with Margin Distributions
Yiding Jiang, Dilip Krishnan, Hossein Mobahi +1
As shown in recent research, deep neural networks can perfectly fit randomly labeled data, but with very poor accuracy on held out data. This phenomenon indicates that loss functio…
Insights on representational similarity in neural networks with canonical correlation
Ari S. Morcos, Maithra Raghu, Samy Bengio
Comparing different neural network representations and determining how representations evolve over time remain challenging open questions in our understanding of the function of ne…
Large Margin Deep Networks for Classification
Gamaleldin F. Elsayed, Dilip Krishnan, Hossein Mobahi +2
We present a formulation of deep learning that aims at producing a large margin classifier. The notion of margin, minimum distance to a decision boundary, has served as the foundat…
Order Matters: Sequence to sequence for sets
Oriol Vinyals, Samy Bengio, Manjunath Kudlur
Sequences have become first class citizens in supervised learning thanks to the resurgence of recurrent neural networks. Many complex tasks that require mapping from or to a sequen…