Flow Matching for Generative Modeling
arXiv:2210.02747
Abstract
We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale. Specifically, we present the notion of Flow Matching (FM), a simulation-free approach for training CNFs based on regressing vector fields of fixed conditional probability paths. Flow Matching is compatible with a general family of Gaussian probability paths for transforming between noise and data samples -- which subsumes existing diffusion paths as specific instances. Interestingly, we find that employing FM with diffusion paths results in a more robust and stable alternative for training diffusion models. Furthermore, Flow Matching opens the door to training CNFs with other, non-diffusion probability paths. An instance of particular interest is using Optimal Transport (OT) displacement interpolation to define the conditional probability paths. These paths are more efficient than diffusion paths, provide faster training and sampling, and result in better generalization. Training CNFs using Flow Matching on ImageNet leads to consistently better performance than alternative diffusion-based methods in terms of both likelihood and sample quality, and allows fast and reliable sample generation using off-the-shelf numerical ODE solvers.
Cited by in corpus (14)
- Mapping Dark Matter in the Milky Way using Normalizing Flows and Gaia DR3
- The Landscape of Unfolding with Machine Learning
- CaloDREAM -- Detector Response Emulation via Attentive flow Matching
- Precision-Machine Learning for the Matrix Element Method
- A Lorentz-Equivariant Transformer for All of the LHC
- Normalizing flows as an enhanced sampling method for atomistic supercooled liquids
- Machine Learning for Estimation and Control of Quantum Systems
- Unraveling the Potential of Diffusion Models in Small Molecule Generation
- Calibrating Bayesian Generative Machine Learning for Bayesiamplification
- Simulation-based Inference for Gravitational-waves from Intermediate-Mass Binary Black Holes in Real Noise
- Generative Unfolding with Distribution Mapping
- Rapid Parameter Estimation for Merging Massive Black Hole Binaries Using Continuous Normalizing Flows
- Encoding off-shell effects in top pair production in Direct Diffusion networks
- Large Language Models -- the Future of Fundamental Physics?