1.6k citations · 11.3k across the 73 of their papers we have counts for
12 papers · 1 filter
Fantastic Generalization Measures and Where to Find Them
Yiding Jiang, Behnam Neyshabur, Hossein Mobahi +2
Generalization of deep networks has been of great interest in recent years, resulting in a number of theoretically and empirically motivated complexity measures. However, most pape…
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
Aniruddh Raghu, Maithra Raghu, Samy Bengio +1
An important research direction in machine learning has centered around developing meta-learning algorithms to tackle few-shot learning. An especially successful algorithm has been…
Memory Based Trajectory-conditioned Policies for Learning from Sparse Rewards
Yijie Guo, Jongwook Choi, Marcin Moczulski +4
Reinforcement learning with sparse rewards is challenging because an agent can rarely obtain non-zero rewards and hence, gradient-based optimization of parameterized policies can b…
A Closed-Form Learned Pooling for Deep Classification Networks
Vighnesh Birodkar, Hossein Mobahi, Dilip Krishnan +1
In modern computer vision tasks, convolutional neural networks (CNNs) are indispensable for image classification tasks due to their efficiency and effectiveness. Part of their supe…
Parallel Scheduled Sampling
Daniel Duckworth, Arvind Neelakantan, Ben Goodrich +2
Auto-regressive models are widely used in sequence generation problems. The output sequence is typically generated in a predetermined order, one discrete unit (pixel or word or cha…
Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
Wei-Lin Chiang, Xuanqing Liu, Si Si +3
Graph convolutional network (GCN) has been successfully applied to many graph-based applications; however, training a large-scale GCN remains challenging. Current SGD-based algorit…
Neural Networks Trained on Natural Scenes Exhibit Gestalt Closure
Been Kim, Emily Reif, Martin Wattenberg +2
The Gestalt laws of perceptual organization, which describe how visual elements in an image are grouped and interpreted, have traditionally been thought of as innate despite their…
Transfusion: Understanding Transfer Learning for Medical Imaging
Maithra Raghu, Chiyuan Zhang, Jon Kleinberg +1
Transfer learning from natural image datasets, particularly ImageNet, using standard large models and corresponding pretrained weights has become a de-facto method for deep learnin…
Identity Crisis: Memorization and Generalization under Extreme Overparameterization
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
We study the interplay between memorization and generalization of overparameterized networks in the extreme case of a single training example and an identity-mapping task. We exami…
Are All Layers Created Equal?
Chiyuan Zhang, Samy Bengio, Yoram Singer
Understanding deep neural networks is a major research objective with notable experimental and theoretical attention in recent years. The practical success of excessively large net…
Semantic Redundancies in Image-Classification Datasets: The 10% You Don't Need
Vighnesh Birodkar, Hossein Mobahi, Samy Bengio
Large datasets have been crucial to the success of deep learning models in the recent years, which keep performing better as they are trained with more labelled data. While there h…
Unsupervised speech representation learning using WaveNet autoencoders
Jan Chorowski, Ron J. Weiss, Samy Bengio +1
We consider the task of unsupervised extraction of meaningful latent representations of speech by applying autoencoding neural networks to speech waveforms. The goal is to learn a…