A multifidelity approach to continual learning for physical systems

Abstract

We introduce a novel continual learning method based on multifidelity deep neural networks. This method learns the correlation between the output of previously trained models and the desired output of the model on the current training dataset, limiting catastrophic forgetting. On its own the multifidelity continual learning method shows robust results that limit forgetting across several datasets. Additionally, we show that the multifidelity method can be combined with existing continual learning methods, including replay and memory aware synapses, to further limit catastrophic forgetting. The proposed continual learning method is especially suited for physical problems where the data satisfy the same physical laws on each domain, or for physics-informed neural networks, because in these cases we expect there to be a strong correlation between the output of the previous model and the model on the current training domain.

References in corpus (15)

• NSFnets (Navier-Stokes Flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations
• A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks
• Physics-informed Neural Networks (PINNs) for Wave Propagation and Full Waveform Inversions
• Efficient training of physics-informed neural networks via importance sampling
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism
• Multilevel domain decomposition-based architectures for physics-informed neural networks
• Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problems
• Recipes for when Physics Fails: Recovering Robust Learning of Physics Informed Neural Networks
• A Multifidelity deep operator network approach to closure for multiscale systems
• On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networks
• Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learning
• Unifying Regularisation Methods for Continual Learning
• Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computing
• Efficient kernel surrogates for neural network-based regression

Cited by in corpus (2)

• Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks
• Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modeling