172 citations · 355 across the 11 of their papers we have counts for
6 papers · 1 filter
Unifying Grokking and Double Descent
Xander Davies, Lauro Langosco, David Krueger
A principled understanding of generalization in deep learning may require unifying disparate observations under a single conceptual framework. Previous work has studied \emph{grokk…
General-purpose, long-context autoregressive modeling with Perceiver AR
Curtis Hawthorne, Andrew Jaegle, Cătălina Cangea +12
Real-world data is high-dimensional: a book, image, or musical performance can easily contain hundreds of thousands of elements even after compression. However, the most commonly u…
Variable-rate discrete representation learning
Sander Dieleman, Charlie Nash, Jesse Engel +1
Semantically meaningful information content in perceptual signals is usually unevenly distributed. In speech signals for example, there are often many silences, and the speed of pr…
Efficient Graph Generation with Graph Recurrent Attention Networks
Renjie Liao, Yujia Li, Yang Song +6
We propose a new family of efficient and expressive deep generative models of graphs, called Graph Recurrent Attention Networks (GRANs). Our model generates graphs one block of nod…
Relational inductive biases, deep learning, and graph networks
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst +24
Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, i…
Autoencoders and Probabilistic Inference with Missing Data: An Exact Solution for The Factor Analysis Case
Christopher K. I. Williams, Charlie Nash, Alfredo Nazábal
Latent variable models can be used to probabilistically "fill-in" missing data entries. The variational autoencoder architecture (Kingma and Welling, 2014; Rezende et al., 2014) in…