Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge

Abstract

We consider the use of Deep Learning methods for modeling complex phenomena like those occurring in natural physical processes. With the large amount of data gathered on these phenomena the data intensive paradigm could begin to challenge more traditional approaches elaborated over the years in fields like maths or physics. However, despite considerable successes in a variety of application domains, the machine learning field is not yet ready to handle the level of complexity required by such problems. Using an example application, namely Sea Surface Temperature Prediction, we show how general background knowledge gained from physics could be used as a guideline for designing efficient Deep Learning models. In order to motivate the approach and to assess its generality we demonstrate a formal link between the solution of a class of differential equations underlying a large family of physical phenomena and the proposed model. Experiments and comparison with series of baselines including a state of the art numerical approach is then provided.

References in corpus (16)

• Auto-Encoding Variational Bayes
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Deep Residual Learning for Image Recognition
• A Recurrent Latent Variable Model for Sequential Data
• FlowNet: Learning Optical Flow with Convolutional Networks
• Deep multi-scale video prediction beyond mean square error
• Accelerating Eulerian Fluid Simulation With Convolutional Networks
• Unsupervised Learning for Physical Interaction through Video Prediction
• Spatio-temporal video autoencoder with differentiable memory
• Deep Kalman Filters
• Transformation-Based Models of Video Sequences
• FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
• Guided Optical Flow Learning
• Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness
• Video Pixel Networks
• Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations

Cited by in corpus (41)

• Learning data driven discretizations for partial differential equations
• Bridging observation, theory and numerical simulation of the ocean using Machine Learning
• On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues
• Tropical Cyclone Track Forecasting using Fused Deep Learning from Aligned Reanalysis Data
• Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting
• Statistical and machine learning ensemble modelling to forecast sea surface temperature
• Physically Explainable CNN for SAR Image Classification
• Towards a Collective Agenda on AI for Earth Science Data Analysis
• Learned discretizations for passive scalar advection in a 2-D turbulent flow
• A Deep Convolutional Neural Network Model for improving WRF Forecasts
• A review of radar-based nowcasting of precipitation and applicable machine learning techniques
• Learning Dynamical Systems from Partial Observations
• Learning Mesh-Based Simulation with Graph Networks
• Deep Integro-Difference Equation Models for Spatio-Temporal Forecasting
• Bayesian neural networks via MCMC: a Python-based tutorial
• Symplectic Learning for Hamiltonian Neural Networks
• Differentiable Programming for Earth System Modeling
• Physics-informed generative neural network: an application to troposphere temperature prediction
• Data-driven Identification of 2D Partial Differential Equations using extracted physical features
• Integrating Domain Knowledge in Data-driven Earth Observation with Process Convolutions
• BAMCAFE: A Bayesian Machine Learning Advanced Forecast Ensemble Method for Complex Turbulent Systems with Partial Observations
• Operator Learning with Neural Fields: Tackling PDEs on General Geometries
• Physics-informed Tensor-train ConvLSTM for Volumetric Velocity Forecasting of Loop Current
• Learning Generalized Quasi-Geostrophic Models Using Deep Neural Numerical Models
• Combining data assimilation and machine learning to infer unresolved scale parametrisation
• Neural Dynamical Systems: Balancing Structure and Flexibility in Physical Prediction
• Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling
• Analyzing Koopman approaches to physics-informed machine learning for long-term sea-surface temperature forecasting
• Data assimilation empowered neural network parameterizations for subgrid processes in geophysical flows
• Deep Neural Networks for Computational Optical Form Measurements
• Deep KKL: Data-driven Output Prediction for Non-Linear Systems
• On the convergence of PINNs
• Guiding Video Prediction with Explicit Procedural Knowledge
• A Deep Learning Approach for Predicting Spatiotemporal Dynamics From Sparsely Observed Data
• Climate Modeling with Neural Diffusion Equations
• Improving Long-term Autoregressive Spatiotemporal Predictions: A Proof of Concept with Fluid Dynamics
• Living in the Physics and Machine Learning Interplay for Earth Observation
• Unraveled Multilevel Transformation Networks for Predicting Sparsely-Observed Spatiotemporal Dynamics
• A Model-Constrained Tangent Slope Learning Approach for Dynamical Systems
• Deep Learning tools to support deforestation monitoring in the Ivory Coast using SAR and Optical satellite imagery
• Data-Driven Continuum Dynamics via Transport-Teleport Duality