Multi-level Convolutional Autoencoder Networks for Parametric Prediction of Spatio-temporal Dynamics

Abstract

A data-driven framework is proposed towards the end of predictive modeling of complex spatio-temporal dynamics, leveraging nested non-linear manifolds. Three levels of neural networks are used, with the goal of predicting the future state of a system of interest in a parametric setting. A convolutional autoencoder is used as the top level to encode the high dimensional input data along spatial dimensions into a sequence of latent variables. A temporal convolutional autoencoder (TCAE) serves as the second level, which further encodes the output sequence from the first level along the temporal dimension, and outputs a set of latent variables that encapsulate the spatio-temporal evolution of the dynamics. The use of dilated temporal convolutions grows the receptive field exponentially with network depth, allowing for efficient processing of long temporal sequences typical of scientific computations. A fully-connected network is used as the third level to learn the mapping between these latent variables and the global parameters from training data, and predict them for new parameters. For future state predictions, the second level uses a temporal convolutional network to predict subsequent steps of the output sequence from the top level. Latent variables at the bottom-most level are decoded to obtain the dynamics in physical space at new global parameters and/or at a future time. Predictive capabilities are evaluated on a range of problems involving discontinuities, wave propagation, strong transients, and coherent structures. The sensitivity of the results to different modeling choices is assessed. The results suggest that given adequate data and careful training, effective data-driven predictive models can be constructed. Perspectives are provided on the present approach and its place in the landscape of model reduction.

References in corpus (24)

• Adam: A Method for Stochastic Optimization
• Semi-Supervised Classification with Graph Convolutional Networks
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• WaveNet: A Generative Model for Raw Audio
• A guide to convolution arithmetic for deep learning
• Language Modeling with Gated Convolutional Networks
• Proper Orthogonal Decomposition Closure Models For Turbulent Flows: A Numerical Comparison
• Data-driven prediction of unsteady flow fields over a circular cylinder using deep learning
• A New Selection Operator for the Discrete Empirical Interpolation Method -- improved a priori error bound and extensions
• Data-Driven POD-Galerkin Reduced Order Model for Turbulent Flows
• Physics-Informed Probabilistic Learning of Linear Embeddings of Non-linear Dynamics With Guaranteed Stability
• Nonlinear Model Order Reduction via Lifting Transformations and Proper Orthogonal Decomposition
• Time-series learning of latent-space dynamics for reduced-order model closure
• Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems
• Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning
• A Convolutional Encoder Model for Neural Machine Translation
• Compressed Convolutional LSTM: An Efficient Deep Learning framework to Model High Fidelity 3D Turbulence
• A Priori Estimation Of Memory Effects In Coarse-Grained Nonlinear Systems Using The Mori-Zwanzig Formalism
• Time-series machine-learning error models for approximate solutions to parameterized dynamical systems
• Model reduction for transport-dominated problems via online adaptive bases and adaptive sampling
• Balanced Truncation Model Reduction for Lifted Nonlinear Systems

Cited by in corpus (38)

• PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain
• Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems
• Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders
• Convolutional neural network based hierarchical autoencoder for nonlinear mode decomposition of fluid field data
• Transformers for Modeling Physical Systems
• Probabilistic neural networks for fluid flow surrogate modeling and data recovery
• On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues
• Convolutional neural networks for fluid flow analysis: toward effective metamodeling and low-dimensionalization
• Sparse identification of nonlinear dynamics with low-dimensionalized flow representations
• Sparsity-promoting algorithms for the discovery of informative Koopman invariant subspaces
• Model fusion with physics-guided machine learning
• An AI-based Domain-Decomposition Non-Intrusive Reduced-Order Model for Extended Domains applied to Multiphase Flow in Pipes
• Towards Reusable Surrogate Models: Graph-Based Transfer Learning on Trusses
• Nonlinear proper orthogonal decomposition for convection-dominated flows
• Geometric deep learning for computational mechanics Part II: Graph embedding for interpretable multiscale plasticity
• Latent-space time evolution of non-intrusive reduced-order models using Gaussian process emulation
• Non-intrusive data-driven reduced-order modeling for time-dependent parametrized problems
• $\text{FastSVD-ML-ROM}$: A Reduced-Order Modeling Framework based on Machine Learning for Real-Time Applications
• Predicting waves in fluids with deep neural network
• Data-driven transient lift attenuation for extreme vortex gust-airfoil interactions
• Adaptive physics-informed neural operator for coarse-grained non-equilibrium flows
• Applying Convolutional Neural Networks to Data on Unstructured Meshes with Space-Filling Curves
• Reduced-order modeling of unsteady fluid flow using neural network ensembles
• Reconstructing three-dimensional bluff body wake from sectional flow fields with convolutional neural networks
• Data Assimilation Predictive GAN (DA-PredGAN): applied to determine the spread of COVID-19
• Deep hybrid model with satellite imagery: how to combine demand modeling and computer vision for behavior analysis?
• Non-intrusive Surrogate Modeling for Parametrized Time-dependent PDEs using Convolutional Autoencoders
• Fast PDE-constrained optimization via self-supervised operator learning
• Conditionally Parameterized, Discretization-Aware Neural Networks for Mesh-Based Modeling of Physical Systems
• Parametric 3D Convolutional Autoencoder for the Prediction of Flow Fields in a Bed Configuration of Hot Particles
• A surrogate model for topology optimisation of elastic structures via parametric autoencoders
• Model order reduction with neural networks: Application to laminar and turbulent flows
• Data-Augmented Predictive Deep Neural Network: Enhancing the extrapolation capabilities of non-intrusive surrogate models
• A discrete physics-informed training for projection-based reduced order models with neural networks
• TAEN: A Model-Constrained Tikhonov Autoencoder Network for Forward and Inverse Problems
• Common kernel-smoothed proper orthogonal decomposition (CKSPOD): An efficient reduced-order model for emulation of spatiotemporally evolving flow dynamics
• Direct data-driven forecast of local turbulent heat flux in Rayleigh-Bénard convection
• Low-dimensional representation of intermittent geophysical turbulence with High-Order Statistics-informed Neural Networks (H-SiNN)