354 citations · 1.9k across the 38 of their papers we have counts for
8 papers · 1 filter
Neural Tangents: Fast and Easy Infinite Neural Networks in Python
Roman Novak, Lechao Xiao, Jiri Hron +4
Neural Tangents is a library designed to enable research into infinite-width neural networks. It provides a high-level API for specifying complex and hierarchical neural network ar…
Thermodynamic Computing
Tom Conte, Erik DeBenedictis, Natesh Ganesh +36
The hardware and software foundations laid in the first half of the 20th Century enabled the computing technologies that have transformed the world, but these foundations are now u…
Information in Infinite Ensembles of Infinitely-Wide Neural Networks
Ravid Shwartz-Ziv, Alexander A. Alemi
In this preliminary work, we study the generalization properties of infinite ensembles of infinitely-wide neural networks. Amazingly, this model family admits tractable calculation…
Variational Predictive Information Bottleneck
Alexander A. Alemi
In classic papers, Zellner demonstrated that Bayesian inference could be derived as the solution to an information theoretic functional. Below we derive a generalized form of this…
On Predictive Information in RNNs
Zhe Dong, Deniz Oktay, Ben Poole +1
Certain biological neurons demonstrate a remarkable capability to optimally compress the history of sensory inputs while being maximally informative about the future. In this work,…
Dueling Decoders: Regularizing Variational Autoencoder Latent Spaces
Bryan Seybold, Emily Fertig, Alex Alemi +1
Variational autoencoders learn unsupervised data representations, but these models frequently converge to minima that fail to preserve meaningful semantic information. For example,…
On Variational Bounds of Mutual Information
Ben Poole, Sherjil Ozair, Aaron van den Oord +2
Estimating and optimizing Mutual Information (MI) is core to many problems in machine learning; however, bounding MI in high dimensions is challenging. To establish tractable and s…
On the Use of ArXiv as a Dataset
Colin B. Clement, Matthew Bierbaum, Kevin P. O'Keeffe +1
The arXiv has collected 1.5 million pre-print articles over 28 years, hosting literature from scientific fields including Physics, Mathematics, and Computer Science. Each pre-print…