Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling

Abstract

Integrating physics models within machine learning models holds considerable promise toward learning robust models with improved interpretability and abilities to extrapolate. In this work, we focus on the integration of incomplete physics models into deep generative models. In particular, we introduce an architecture of variational autoencoders (VAEs) in which a part of the latent space is grounded by physics. A key technical challenge is to strike a balance between the incomplete physics and trainable components such as neural networks for ensuring that the physics part is used in a meaningful manner. To this end, we propose a regularized learning method that controls the effect of the trainable components and preserves the semantics of the physics-based latent variables as intended. We not only demonstrate generative performance improvements over a set of synthetic and real-world datasets, but we also show that we learn robust models that can consistently extrapolate beyond the training distribution in a meaningful manner. Moreover, we show that we can control the generative process in an interpretable manner.

References in corpus (46)

• Auto-Encoding Variational Bayes
• Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data
• B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data
• The frontier of simulation-based inference
• Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
• A Recurrent Latent Variable Model for Sequential Data
• PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
• Hamiltonian Neural Networks
• Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge
• Data-assisted reduced-order modeling of extreme events in complex dynamical systems
• Deep learning of multi-element abundances from high-resolution spectroscopic data
• Kohn-Sham equations as regularizer: building prior knowledge into machine-learned physics
• Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers
• Hybrid FEM-NN models: Combining artificial neural networks with the finite element method
• Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting
• ODE${}^2$VAE: Deep generative second order ODEs with Bayesian neural networks
• Solving inverse-PDE problems with physics-aware neural networks
• Lagrangian Neural Networks
• Physics-informed deep generative models
• Neural Network Augmented Physics Models for Systems with Partially Unknown Dynamics: Application to Slider-Crank Mechanism
• Embedding Hard Physical Constraints in Neural Network Coarse-Graining of 3D Turbulence
• Grey-box models for wave loading prediction
• Blending Diverse Physical Priors with Neural Networks
• Incorporating physical constraints in a deep probabilistic machine learning framework for coarse-graining dynamical systems
• The Information Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Models
• Explainable Machine Learning with Prior Knowledge: An Overview
• SimGANs: Simulator-Based Generative Adversarial Networks for ECG Synthesis to Improve Deep ECG Classification
• HybridNet: Integrating Model-based and Data-driven Learning to Predict Evolution of Dynamical Systems
• Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks
• A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables
• Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progression
• Data-Augmented Contact Model for Rigid Body Simulation
• Enforcing Deterministic Constraints on Generative Adversarial Networks for Emulating Physical Systems
• TossingBot: Learning to Throw Arbitrary Objects with Residual Physics
• Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video
• SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning
• Neural Dynamical Systems: Balancing Structure and Flexibility in Physical Prediction
• AdjointNet: Constraining machine learning models with physics-based codes
• Hybrid Physical-Deep Learning Model for Astronomical Inverse Problems
• Thermodynamic Consistent Neural Networks for Learning Material Interfacial Mechanics
• Living in the Physics and Machine Learning Interplay for Earth Observation
• Interpretable machine learning models: a physics-based view
• Variational Autoencoding of PDE Inverse Problems
• Physics-aware, probabilistic model order reduction with guaranteed stability
• Amortized Synthesis of Constrained Configurations Using a Differentiable Surrogate
• Embedded-model flows: Combining the inductive biases of model-free deep learning and explicit probabilistic modeling