249 citations · 398 across the 7 of their papers we have counts for
5 papers · 1 filter
Vector Quantized Models for Planning
Sherjil Ozair, Yazhe Li, Ali Razavi +3
Recent developments in the field of model-based RL have proven successful in a range of environments, especially ones where planning is essential. However, such successes have been…
Do Transformers Need Deep Long-Range Memory
Jack W. Rae, Ali Razavi
Deep attention models have advanced the modelling of sequential data across many domains. For language modelling in particular, the Transformer-XL -- a Transformer augmented with a…
Generating Diverse High-Fidelity Images with VQ-VAE-2
Ali Razavi, Aaron van den Oord, Oriol Vinyals
We explore the use of Vector Quantized Variational AutoEncoder (VQ-VAE) models for large scale image generation. To this end, we scale and enhance the autoregressive priors used in…
Preventing Posterior Collapse with delta-VAEs
Ali Razavi, Aäron van den Oord, Ben Poole +1
Due to the phenomenon of "posterior collapse," current latent variable generative models pose a challenging design choice that either weakens the capacity of the decoder or require…
Population Based Training of Neural Networks
Max Jaderberg, Valentin Dalibard, Simon Osindero +9
Neural networks dominate the modern machine learning landscape, but their training and success still suffer from sensitivity to empirical choices of hyperparameters such as model a…