Flows for simultaneous manifold learning and density estimation

Abstract

We introduce manifold-learning flows (M-flows), a new class of generative models that simultaneously learn the data manifold as well as a tractable probability density on that manifold. Combining aspects of normalizing flows, GANs, autoencoders, and energy-based models, they have the potential to represent datasets with a manifold structure more faithfully and provide handles on dimensionality reduction, denoising, and out-of-distribution detection. We argue why such models should not be trained by maximum likelihood alone and present a new training algorithm that separates manifold and density updates. In a range of experiments we demonstrate how M-flows learn the data manifold and allow for better inference than standard flows in the ambient data space.

arXiv commentCode at https://github.com/johannbrehmer/manifold-flow , v2: multiple new experiments, v3: added comparison with probabilistic auto-encoder

References in corpus (12)

• The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations
• A Brief Introduction to PYTHIA 8.1
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• DELPHES 3, A modular framework for fast simulation of a generic collider experiment
• Mining gold from implicit models to improve likelihood-free inference
• Approximating Likelihood Ratios with Calibrated Discriminative Classifiers
• Interpolating between Optimal Transport and MMD using Sinkhorn Divergences
• Are GANs Created Equal? A Large-Scale Study
• Likelihood-free inference with emulator networks
• Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling
• Invertible Autoencoder for domain adaptation
• Backpropagation for Implicit Spectral Densities

Cited by in corpus (22)

• Getting High: High Fidelity Simulation of High Granularity Calorimeters with High Speed
• Nested Sampling with Normalising Flows for Gravitational-Wave Inference
• Understanding Event-Generation Networks via Uncertainties
• Quasi Anomalous Knowledge: Searching for new physics with embedded knowledge
• SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds
• The LHC Olympics 2020: A Community Challenge for Anomaly Detection in High Energy Physics
• Probabilistic Autoencoder
• Neural Manifold Ordinary Differential Equations
• The Power Spherical distribution
• RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior
• Flow-Based Likelihoods for Non-Gaussian Inference
• Latent Variable Modelling with Hyperbolic Normalizing Flows
• Trumpets: Injective Flows for Inference and Inverse Problems
• Normalizing Flows Across Dimensions
• Generalized Energy Based Models
• Principled Interpolation in Normalizing Flows
• Latent Space Refinement for Deep Generative Models
• Measure Transport with Kernel Stein Discrepancy
• Autoencoding Under Normalization Constraints
• Flow-based Generative Models for Learning Manifold to Manifold Mappings
• Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows
• Ordering Dimensions with Nested Dropout Normalizing Flows