Publications (52)
Rolling Diffusion Models
David Ruhe, Jonathan Heek, Tim Salimans +1
Diffusion models have recently been increasingly applied to temporal data such as video, fluid mechanics simulations, or climate data. These methods generally treat subsequent fram…
Variational Lossy Autoencoder
Xi Chen, Diederik P. Kingma, Tim Salimans +5
Representation learning seeks to expose certain aspects of observed data in a learned representation that's amenable to downstream tasks like classification. For instance, a good r…
Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMD
Emiel Hoogeboom, David Ruhe, Jonathan Heek +2
It is currently difficult to distill discrete diffusion models. In contrast, continuous diffusion literature has many distillation approaches methods that can reduce sampling steps…
MetNet: A Neural Weather Model for Precipitation Forecasting
Casper Kaae Sønderby, Lasse Espeholt, Jonathan Heek +6
Weather forecasting is a long standing scientific challenge with direct social and economic impact. The task is suitable for deep neural networks due to vast amounts of continuousl…
Imagen Video: High Definition Video Generation with Diffusion Models
Jonathan Ho, William Chan, Chitwan Saharia +8
We present Imagen Video, a text-conditional video generation system based on a cascade of video diffusion models. Given a text prompt, Imagen Video generates high definition videos…
Milking CowMask for Semi-Supervised Image Classification
Geoff French, Avital Oliver, Tim Salimans
Consistency regularization is a technique for semi-supervised learning that underlies a number of strong results for classification with few labeled data. It works by encouraging a…
Variational Dropout and the Local Reparameterization Trick
Diederik P. Kingma, Tim Salimans, Max Welling
We investigate a local reparameterizaton technique for greatly reducing the variance of stochastic gradients for variational Bayesian inference (SGVB) of a posterior over model par…
Multistep Distillation of Diffusion Models via Moment Matching
Tim Salimans, Thomas Mensink, Jonathan Heek +1
We present a new method for making diffusion models faster to sample. The method distills many-step diffusion models into few-step models by matching conditional expectations of th…
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
Tim Salimans, Jonathan Ho, Xi Chen +2
We explore the use of Evolution Strategies (ES), a class of black box optimization algorithms, as an alternative to popular MDP-based RL techniques such as Q-learning and Policy Gr…
Variational Diffusion Models
Diederik P. Kingma, Tim Salimans, Ben Poole +1
Diffusion-based generative models have demonstrated a capacity for perceptually impressive synthesis, but can they also be great likelihood-based models? We answer this in the affi…
Hydra: Preserving Ensemble Diversity for Model Distillation
Linh Tran, Bastiaan S. Veeling, Kevin Roth +7
Ensembles of models have been empirically shown to improve predictive performance and to yield robust measures of uncertainty. However, they are expensive in computation and memory…
PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications
Tim Salimans, Andrej Karpathy, Xi Chen +1
PixelCNNs are a recently proposed class of powerful generative models with tractable likelihood. Here we discuss our implementation of PixelCNNs which we make available at https://…
Imagen 3
Imagen-Team-Google, :, Jason Baldridge +257
We introduce Imagen 3, a latent diffusion model that generates high quality images from text prompts. We describe our quality and responsibility evaluations. Imagen 3 is preferred…
Agent-Centric Representations for Multi-Agent Reinforcement Learning
Wenling Shang, Lasse Espeholt, Anton Raichuk +1
Object-centric representations have recently enabled significant progress in tackling relational reasoning tasks. By building a strong object-centric inductive bias into neural arc…
Video Diffusion Models
Jonathan Ho, Tim Salimans, Alexey Gritsenko +3
Generating temporally coherent high fidelity video is an important milestone in generative modeling research. We make progress towards this milestone by proposing a diffusion model…
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
Tim Salimans, Diederik P. Kingma
We present weight normalization: a reparameterization of the weight vectors in a neural network that decouples the length of those weight vectors from their direction. By reparamet…
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
Chitwan Saharia, William Chan, Saurabh Saxena +11
We present Imagen, a text-to-image diffusion model with an unprecedented degree of photorealism and a deep level of language understanding. Imagen builds on the power of large tran…
A Structured Variational Auto-encoder for Learning Deep Hierarchies of Sparse Features
Tim Salimans
In this note we present a generative model of natural images consisting of a deep hierarchy of layers of latent random variables, each of which follows a new type of distribution t…
Dota 2 with Large Scale Deep Reinforcement Learning
OpenAI, :, Christopher Berner +24
On April 13th, 2019, OpenAI Five became the first AI system to defeat the world champions at an esports game. The game of Dota 2 presents novel challenges for AI systems such as lo…
The Likelihood of Mixed Hitting Times
Jaap H. Abbring, Tim Salimans
We present a method for computing the likelihood of a mixed hitting-time model that specifies durations as the first time a latent Lévy process crosses a heterogeneous threshold.…
Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
Tim Salimans, Diederik P. Kingma, Max Welling
Recent advances in stochastic gradient variational inference have made it possible to perform variational Bayesian inference with posterior approximations containing auxiliary rand…
Cascaded Diffusion Models for High Fidelity Image Generation
Jonathan Ho, Chitwan Saharia, William Chan +3
We show that cascaded diffusion models are capable of generating high fidelity images on the class-conditional ImageNet generation benchmark, without any assistance from auxiliary…
EM Distillation for One-step Diffusion Models
Sirui Xie, Zhisheng Xiao, Diederik P Kingma +6
While diffusion models can learn complex distributions, sampling requires a computationally expensive iterative process. Existing distillation methods enable efficient sampling, bu…
Progressive Distillation for Fast Sampling of Diffusion Models
Tim Salimans, Jonathan Ho
Diffusion models have recently shown great promise for generative modeling, outperforming GANs on perceptual quality and autoregressive models at density estimation. A remaining do…
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our…
Multistep Consistency Models
Jonathan Heek, Emiel Hoogeboom, Tim Salimans
Diffusion models are relatively easy to train but require many steps to generate samples. Consistency models are far more difficult to train, but generate samples in a single step.…
Autoregressive Diffusion Models
Emiel Hoogeboom, Alexey A. Gritsenko, Jasmijn Bastings +3
We introduce Autoregressive Diffusion Models (ARDMs), a model class encompassing and generalizing order-agnostic autoregressive models (Uria et al., 2014) and absorbing discrete di…
Blurring Diffusion Models
Emiel Hoogeboom, Tim Salimans
Recently, Rissanen et al., (2022) have presented a new type of diffusion process for generative modeling based on heat dissipation, or blurring, as an alternative to isotropic Gaus…
Simple diffusion: End-to-end diffusion for high resolution images
Emiel Hoogeboom, Jonathan Heek, Tim Salimans
Currently, applying diffusion models in pixel space of high resolution images is difficult. Instead, existing approaches focus on diffusion in lower dimensional spaces (latent diff…
Policy Gradient Search: Online Planning and Expert Iteration without Search Trees
Thomas Anthony, Robert Nishihara, Philipp Moritz +2
Monte Carlo Tree Search (MCTS) algorithms perform simulation-based search to improve policies online. During search, the simulation policy is adapted to explore the most promising…
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression
Tim Salimans, David A. Knowles
We propose a general algorithm for approximating nonstandard Bayesian posterior distributions. The algorithm minimizes the Kullback-Leibler divergence of an approximating distribut…
Observing Dark Worlds: A crowdsourcing experiment for dark matter mapping
David Harvey, Thomas D. Kitching, Joyce Noah-Vanhoucke +2
We present the results and conclusions from the citizen science competition `Observing Dark Worlds', where we asked participants to calculate the positions of dark matter halos fro…
Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network
Grigory Bartosh, David Ruhe, Emiel Hoogeboom +3
Diffusion models achieve state-of-the-art generative performance but suffer from high computational costs during inference due to the repeated evaluation of a heavy neural network.…
Unified Latents (UL): How to train your latents
Jonathan Heek, Emiel Hoogeboom, Thomas Mensink +1
We present Unified Latents (UL), a framework for learning latent representations that are jointly regularized by a diffusion prior and decoded by a diffusion model. By linking the…
Improving GANs Using Optimal Transport
Tim Salimans, Han Zhang, Alec Radford +1
We present Optimal Transport GAN (OT-GAN), a variant of generative adversarial nets minimizing a new metric measuring the distance between the generator distribution and the data d…
IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
Rianne van den Berg, Alexey A. Gritsenko, Mostafa Dehghani +2
In this paper we analyse and improve integer discrete flows for lossless compression. Integer discrete flows are a recently proposed class of models that learn invertible transform…
How Good is the Bayes Posterior in Deep Neural Networks Really?
Florian Wenzel, Kevin Roth, Bastiaan S. Veeling +7
During the past five years the Bayesian deep learning community has developed increasingly accurate and efficient approximate inference procedures that allow for Bayesian inference…
Classifier-Free Diffusion Guidance
Jonathan Ho, Tim Salimans
Classifier guidance is a recently introduced method to trade off mode coverage and sample fidelity in conditional diffusion models post training, in the same spirit as low temperat…
Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion
Emiel Hoogeboom, Thomas Mensink, Jonathan Heek +3
Latent diffusion models have become the popular choice for scaling up diffusion models for high resolution image synthesis. Compared to pixel-space models that are trained end-to-e…
Improving Variational Inference with Inverse Autoregressive Flow
Diederik P. Kingma, Tim Salimans, Rafal Jozefowicz +3
The framework of normalizing flows provides a general strategy for flexible variational inference of posteriors over latent variables. We propose a new type of normalizing flow, in…
On Using Control Variates with Stochastic Approximation for Variational Bayes and its Connection to Stochastic Linear Regression
Tim Salimans, David A. Knowles
Recently, we and several other authors have written about the possibilities of using stochastic approximation techniques for fitting variational approximations to intractable Bayes…
Covariance-aware sampling for Diffusion Models
Andrea Schioppa, Tim Salimans
We present a covariance-aware sampler that improves the quality of pixel-space Diffusion Model (DM) sampling in the few-step regime. We hypothesize that in the few-step regime samp…
The k-tied Normal Distribution: A Compact Parameterization of Gaussian Mean Field Posteriors in Bayesian Neural Networks
Jakub Swiatkowski, Kevin Roth, Bastiaan S. Veeling +7
Variational Bayesian Inference is a popular methodology for approximating posterior distributions over Bayesian neural network weights. Recent work developing this class of methods…
Palette: Image-to-Image Diffusion Models
Chitwan Saharia, William Chan, Huiwen Chang +5
This paper develops a unified framework for image-to-image translation based on conditional diffusion models and evaluates this framework on four challenging image-to-image transla…
Image Super-Resolution via Iterative Refinement
Chitwan Saharia, Jonathan Ho, William Chan +3
We present SR3, an approach to image Super-Resolution via Repeated Refinement. SR3 adapts denoising diffusion probabilistic models to conditional image generation and performs supe…
Learning Montezuma's Revenge from a Single Demonstration
Tim Salimans, Richard Chen
We propose a new method for learning from a single demonstration to solve hard exploration tasks like the Atari game Montezuma's Revenge. Instead of imitating human demonstrations,…
Axial Attention in Multidimensional Transformers
Jonathan Ho, Nal Kalchbrenner, Dirk Weissenborn +1
We propose Axial Transformers, a self-attention-based autoregressive model for images and other data organized as high dimensional tensors. Existing autoregressive models either su…
Lossy Compression with Gaussian Diffusion
Lucas Theis, Tim Salimans, Matthew D. Hoffman +1
We consider a novel lossy compression approach based on unconditional diffusion generative models, which we call DiffC. Unlike modern compression schemes which rely on transform co…
A Spectral Energy Distance for Parallel Speech Synthesis
Alexey A. Gritsenko, Tim Salimans, Rianne van den Berg +2
Speech synthesis is an important practical generative modeling problem that has seen great progress over the last few years, with likelihood-based autoregressive neural models now…
On Distillation of Guided Diffusion Models
Chenlin Meng, Robin Rombach, Ruiqi Gao +4
Classifier-free guided diffusion models have recently been shown to be highly effective at high-resolution image generation, and they have been widely used in large-scale diffusion…
Implementing and Automating Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression
Tim Salimans
We recently proposed a general algorithm for approximating nonstandard Bayesian posterior distributions by minimization of their Kullback-Leibler divergence with respect to a more…
Improved Techniques for Training GANs
Tim Salimans, Ian Goodfellow, Wojciech Zaremba +3
We present a variety of new architectural features and training procedures that we apply to the generative adversarial networks (GANs) framework. We focus on two applications of GA…