Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Abstract

A central problem in machine learning involves modeling complex data-sets using highly flexible families of probability distributions in which learning, sampling, inference, and evaluation are still analytically or computationally tractable. Here, we develop an approach that simultaneously achieves both flexibility and tractability. The essential idea, inspired by non-equilibrium statistical physics, is to systematically and slowly destroy structure in a data distribution through an iterative forward diffusion process. We then learn a reverse diffusion process that restores structure in data, yielding a highly flexible and tractable generative model of the data. This approach allows us to rapidly learn, sample from, and evaluate probabilities in deep generative models with thousands of layers or time steps, as well as to compute conditional and posterior probabilities under the learned model. We additionally release an open source reference implementation of the algorithm.

References in corpus (8)

• Loopy Belief Propagation for Approximate Inference: An Empirical Study
• NICE: Non-linear Independent Components Estimation
• A note on the evaluation of generative models
• Better Mixing via Deep Representations
• Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition
• High-Dimensional Probability Estimation with Deep Density Models
• Deep Directed Generative Autoencoders
• On the Equivalence Between Deep NADE and Generative Stochastic Networks

Cited by in corpus (145)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• An Introduction to Variational Autoencoders
• On the Opportunities and Risks of Foundation Models
• Diffusion Models Beat GANs on Image Synthesis
• Diffusion Models in Vision: A Survey
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Normalizing Flows: An Introduction and Review of Current Methods
• Blended Diffusion for Text-driven Editing of Natural Images
• Improved Denoising Diffusion Probabilistic Models
• Deep Learning for Time Series Forecasting: Tutorial and Literature Survey
• The CosmoVerse White Paper: Addressing observational tensions in cosmology with systematics and fundamental physics
• Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review
• Large AI Models in Health Informatics: Applications, Challenges, and the Future
• Optimizing the Latent Space of Generative Networks
• Improving Variational Inference with Inverse Autoregressive Flow
• Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art
• SpaText: Spatio-Textual Representation for Controllable Image Generation
• DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
• VideoGPT: Video Generation using VQ-VAE and Transformers
• A full-stack view of probabilistic computing with p-bits: devices, architectures and algorithms
• StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation
• AI Art in Architecture
• DiffWave: A Versatile Diffusion Model for Audio Synthesis
• Generative AI for Synthetic Data Across Multiple Medical Modalities: A Systematic Review of Recent Developments and Challenges
• Diffusion Models, Image Super-Resolution And Everything: A Survey
• Perspective: Energy Landscapes for Machine Learning
• Time Series Diffusion Method: A Denoising Diffusion Probabilistic Model for Vibration Signal Generation
• Denoising Diffusion Implicit Models
• A survey of recent methods for addressing AI fairness and bias in biomedicine
• SpectralDiff: A Generative Framework for Hyperspectral Image Classification with Diffusion Models
• DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics
• Super-Resolution with Deep Convolutional Sufficient Statistics
• Taming the Power of Diffusion Models for High-Quality Virtual Try-On with Appearance Flow
• LDMVFI: Video Frame Interpolation with Latent Diffusion Models
• Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed
• Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling
• From data to noise to data: mixing physics across temperatures with generative artificial intelligence
• On Fast Sampling of Diffusion Probabilistic Models
• ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis
• Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges
• NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Rates
• medigan: a Python library of pretrained generative models for medical image synthesis
• Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting
• Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech
• SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows
• Learning Hierarchical Features from Generative Models
• Con-CDVAE: A method for the conditional generation of crystal structures
• Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpoint
• Adversarial score matching and improved sampling for image generation
• Improved Contrastive Divergence Training of Energy Based Models
• DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation
• SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models
• Training Recurrent Neural Networks by Diffusion
• Diffusion Probabilistic Models for 3D Point Cloud Generation
• Generalizing Hamiltonian Monte Carlo with Neural Networks
• Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective
• Ensemble flow reconstruction in the atmospheric boundary layer from spatially limited measurements through latent diffusion models
• Deep Learning for Optical Tweezers
• A Domain Translation Framework with an Adversarial Denoising Diffusion Model to Generate Synthetic Datasets of Echocardiography Images
• Can AI be enabled to dynamical downscaling? A Latent Diffusion Model to mimic km-scale COSMO5.0\_CLM9 simulations
• Transport Analysis of Infinitely Deep Neural Network
• Bayesian Renormalization
• Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution
• Fast Diffusion EM: a diffusion model for blind inverse problems with application to deconvolution
• Review of end-to-end speech synthesis technology based on deep learning
• Score-Based Generative Models for PET Image Reconstruction
• Diffusion Models for Tabular Data Imputation and Synthetic Data Generation
• DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM
• Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time-Series
• How Much is Enough? A Study on Diffusion Times in Score-based Generative Models
• Stochastic Normalizing Flows
• Machine-Learning-Assisted Photonic Device Development: A Multiscale Approach from Theory to Characterization
• Learning Energy-Based Models by Diffusion Recovery Likelihood
• Counterfactual Explanations for Medical Image Classification and Regression using Diffusion Autoencoder
• Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net
• LightGrad: Lightweight Diffusion Probabilistic Model for Text-to-Speech
• Treatment-aware Diffusion Probabilistic Model for Longitudinal MRI Generation and Diffuse Glioma Growth Prediction
• License Plate Super-Resolution Using Diffusion Models
• Bilateral Denoising Diffusion Models
• Inference in Deep Networks in High Dimensions
• Training Deep Energy-Based Models with f-Divergence Minimization
• Normalizing flows as an enhanced sampling method for atomistic supercooled liquids
• The Inverse of Exact Renormalization Group Flows as Statistical Inference
• Integrating Dynamical Systems Modeling with Spatiotemporal scRNA-seq Data Analysis
• Annealed Generative Adversarial Networks
• Diffusion-Augmented Depth Prediction with Sparse Annotations
• Riemannian Denoising Diffusion Probabilistic Models
• Reflection-Equivariant Diffusion for 3D Structure Determination from Isotopologue Rotational Spectra in Natural Abundance
• LaMD: Latent Motion Diffusion for Image-Conditional Video Generation
• Generative Adversarial Networks with Decoder-Encoder Output Noise
• Conditional Image Generation with Score-Based Diffusion Models
• Zero-Shot Paragraph-level Handwriting Imitation with Latent Diffusion Models
• Global Search for Optimal Low Thrust Spacecraft Trajectories using Diffusion Models and the Indirect Method
• PROUD: PaRetO-gUided Diffusion Model for Multi-objective Generation
• Physics-informed generative model for drug-like molecule conformers
• TrajDiffuse: A Conditional Diffusion Model for Environment-Aware Trajectory Prediction
• Unsupervised Multi-Domain Image Translation with Domain-Specific Encoders/Decoders
• AI-generated art perceptions with GenFrame -- an image-generating picture frame
• Automatic Relevance Determination For Deep Generative Models
• PixelCNN Models with Auxiliary Variables for Natural Image Modeling
• Gaussian Gated Linear Networks
• SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models
• RGB-D-Fusion: Image Conditioned Depth Diffusion of Humanoid Subjects
• Inferring Hidden Symmetries of Exotic Magnets from Detecting Explicit Order Parameters
• Synthetic Tabular Data: Methods, Attacks and Defenses
• Unlearning regularization for Boltzmann Machines
• Improved Autoregressive Modeling with Distribution Smoothing
• Generative Unfolding with Distribution Mapping
• Generative Design of a Gas Turbine Combustor Using Invertible Neural Networks
• Stochastic Super-resolution of Cosmological Simulations with Denoising Diffusion Models
• Dreaming of Electrical Waves: Generative Modeling of Cardiac Excitation Waves using Diffusion Models
• A Mathematical Framework for Learning Probability Distributions
• Zero Shot Molecular Generation via Similarity Kernels
• Generating Multiphase Fluid Configurations in Fractures using Diffusion Models
• Fokker-Planck Score Learning: Efficient Free-Energy Estimation under Periodic Boundary Conditions
• Harmonic Path Integral Diffusion
• Symmetries and control in generative neural nets
• Large, fast and accurate HI intensity maps with latent overlap diffusion
• Language Guided Adversarial Purification
• Diffusion models for Handwriting Generation
• Detecting Modeling Bias with Continuous Time Flow Models on Weak Lensing Maps
• D2C: Diffusion-Denoising Models for Few-shot Conditional Generation
• Explaining the effects of non-convergent sampling in the training of Energy-Based Models
• U-Turn Diffusion
• Bridge Diffusion Model: Bridge Chinese Text-to-Image Diffusion Model with English Communities
• Image Augmentation with Controlled Diffusion for Weakly-Supervised Semantic Segmentation
• torchdistill Meets Hugging Face Libraries for Reproducible, Coding-Free Deep Learning Studies: A Case Study on NLP
• Choose Your Diffusion: Efficient and flexible ways to accelerate the diffusion model in fast high energy physics simulation
• Turbulent Injection assisted by Diffusion Models for Scale Resolving Simulations
• Monte Carlo guided Diffusion for Bayesian linear inverse problems
• Variational Auto-Decoder: A Method for Neural Generative Modeling from Incomplete Data
• Note on Equivalence Between Recurrent Neural Network Time Series Models and Variational Bayesian Models
• One-shot Unsupervised Domain Adaptation with Personalized Diffusion Models
• Rebooting Neuromorphic Hardware Design -- A Complexity Engineering Approach
• The Wilson Machine for Image Modeling
• RBM-Flow and D-Flow: Invertible Flows with Discrete Energy Base Spaces
• Neural Differential Appearance Equations
• Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models
• Latent Transformations for Discrete-Data Normalising Flows
• Fixed-kinetic Neural Hamiltonian Flows for enhanced interpretability and reduced complexity
• Lane Graph Extraction from Aerial Imagery via Lane Segmentation Refinement with Diffusion Models
• Image Disguise based on Generative Model
• Theoretical and Empirical Analysis of a Fast Algorithm for Extracting Polygons from Signed Distance Bounds
• Beyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic Models
• Hamiltonian Dynamics with Non-Newtonian Momentum for Rapid Sampling