Multi-scale Time-stepping of Partial Differential Equations with Transformers
arXiv:2311.02225 · doi:10.1016/j.cma.2024.116983
Abstract
Developing fast surrogates for Partial Differential Equations (PDEs) will accelerate design and optimization in almost all scientific and engineering applications. Neural networks have been receiving ever-increasing attention and demonstrated remarkable success in computational modeling of PDEs, however; their prediction accuracy is not at the level of full deployment. In this work, we utilize the transformer architecture, the backbone of numerous state-of-the-art AI models, to learn the dynamics of physical systems as the mixing of spatial patterns learned by a convolutional autoencoder. Moreover, we incorporate the idea of multi-scale hierarchical time-stepping to increase the prediction speed and decrease accumulated error over time. Our model achieves similar or better results in predicting the time-evolution of Navier-Stokes equations compared to the powerful Fourier Neural Operator (FNO) and two transformer-based neural operators OFormer and Galerkin Transformer.
References in corpus (6)
- Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networks
- Multiwavelet-based Operator Learning for Differential Equations
- Learning Operators with Coupled Attention
- GNOT: A General Neural Operator Transformer for Operator Learning
- Towards Multi-spatiotemporal-scale Generalized PDE Modeling
- Generalizable data-driven turbulence closure modeling on unstructured grids with differentiable physics