172 citations · 355 across the 11 of their papers we have counts for
12 papers
Unifying Grokking and Double Descent
Xander Davies, Lauro Langosco, David Krueger
A principled understanding of generalization in deep learning may require unifying disparate observations under a single conceptual framework. Previous work has studied \emph{grokk…
Transframer: Arbitrary Frame Prediction with Generative Models
Charlie Nash, João Carreira, Jacob Walker +4
We present a general-purpose framework for image modelling and vision tasks based on probabilistic frame prediction. Our approach unifies a broad range of tasks, from image segment…
General-purpose, long-context autoregressive modeling with Perceiver AR
Curtis Hawthorne, Andrew Jaegle, Cătălina Cangea +12
Real-world data is high-dimensional: a book, image, or musical performance can easily contain hundreds of thousands of elements even after compression. However, the most commonly u…
HDMapGen: A Hierarchical Graph Generative Model of High Definition Maps
Lu Mi, Hang Zhao, Charlie Nash +7
High Definition (HD) maps are maps with precise definitions of road lanes with rich semantics of the traffic rules. They are critical for several key stages in an autonomous drivin…
Variable-rate discrete representation learning
Sander Dieleman, Charlie Nash, Jesse Engel +1
Semantically meaningful information content in perceptual signals is usually unevenly distributed. In speech signals for example, there are often many silences, and the speed of pr…
Generating Images with Sparse Representations
Charlie Nash, Jacob Menick, Sander Dieleman +1
The high dimensionality of images presents architecture and sampling-efficiency challenges for likelihood-based generative models. Previous approaches such as VQ-VAE use deep autoe…
PolyGen: An Autoregressive Generative Model of 3D Meshes
Charlie Nash, Yaroslav Ganin, S. M. Ali Eslami +1
Polygon meshes are an efficient representation of 3D geometry, and are of central importance in computer graphics, robotics and games development. Existing learning-based approache…
Efficient Graph Generation with Graph Recurrent Attention Networks
Renjie Liao, Yujia Li, Yang Song +6
We propose a new family of efficient and expressive deep generative models of graphs, called Graph Recurrent Attention Networks (GRANs). Our model generates graphs one block of nod…
Autoregressive Energy Machines
Charlie Nash, Conor Durkan
Neural density estimators are flexible families of parametric models which have seen widespread use in unsupervised machine learning in recent years. Maximum-likelihood training ty…
Relational inductive biases, deep learning, and graph networks
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst +24
Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, i…
Inverting Supervised Representations with Autoregressive Neural Density Models
Charlie Nash, Nate Kushman, Christopher K. I. Williams
We present a method for feature interpretation that makes use of recent advances in autoregressive density estimation models to invert model representations. We train generative in…
Autoencoders and Probabilistic Inference with Missing Data: An Exact Solution for The Factor Analysis Case
Christopher K. I. Williams, Charlie Nash, Alfredo Nazábal
Latent variable models can be used to probabilistically "fill-in" missing data entries. The variational autoencoder architecture (Kingma and Welling, 2014; Rezende et al., 2014) in…