1.4k citations · 8.8k across the 91 of their papers we have counts for
10 papers · 1 filter
Improved generator objectives for GANs
Ben Poole, Alexander A. Alemi, Jascha Sohl-Dickstein +1
We present a framework to understand GAN training as alternating density ratio estimation and approximate divergence minimization. This provides an interpretation for the mismatche…
Survey of Expressivity in Deep Neural Networks
Maithra Raghu, Ben Poole, Jon Kleinberg +2
We survey results on neural network expressivity described in "On the Expressive Power of Deep Neural Networks". The paper motivates and develops three natural measures of expressi…
Capacity and Trainability in Recurrent Neural Networks
Jasmine Collins, Jascha Sohl-Dickstein, David Sussillo
Two potential bottlenecks on the expressiveness of recurrent neural networks (RNNs) are their ability to store information about the task in their parameters, and to store informat…
Input Switched Affine Networks: An RNN Architecture Designed for Interpretability
Jakob N. Foerster, Justin Gilmer, Jan Chorowski +2
There exist many problem domains where the interpretability of neural network models is essential for deployment. Here we introduce a recurrent architecture composed of input-switc…
Unrolled Generative Adversarial Networks
Luke Metz, Ben Poole, David Pfau +1
We introduce a method to stabilize Generative Adversarial Networks (GANs) by defining the generator objective with respect to an unrolled optimization of the discriminator. This al…
Deep Information Propagation
Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli +1
We study the behavior of untrained neural networks whose weights and biases are randomly distributed using mean field theory. We show the existence of depth scales that naturally l…
Exponential expressivity in deep neural networks through transient chaos
Ben Poole, Subhaneil Lahiri, Maithra Raghu +2
We combine Riemannian geometry with the mean field theory of high dimensional chaos to study the nature of signal propagation in generic, deep neural networks with random weights.…
On the Expressive Power of Deep Neural Networks
Maithra Raghu, Ben Poole, Jon Kleinberg +2
We propose a new approach to the problem of neural network expressivity, which seeks to characterize how structural properties of a neural network family affect the functions it is…
Density estimation using Real NVP
Laurent Dinh, Jascha Sohl-Dickstein, Samy Bengio
Unsupervised learning of probabilistic models is a central yet challenging problem in machine learning. Specifically, designing models with tractable learning, sampling, inference…
A universal tradeoff between power, precision and speed in physical communication
Subhaneil Lahiri, Jascha Sohl-Dickstein, Surya Ganguli
Maximizing the speed and precision of communication while minimizing power dissipation is a fundamental engineering design goal. Also, biological systems achieve remarkable speed,…