Multi-Grid Graph Neural Networks with Self-Attention for Computational Mechanics

Abstract

Advancement in finite element methods have become essential in various disciplines, and in particular for Computational Fluid Dynamics (CFD), driving research efforts for improved precision and efficiency. While Convolutional Neural Networks (CNNs) have found success in CFD by mapping meshes into images, recent attention has turned to leveraging Graph Neural Networks (GNNs) for direct mesh processing. This paper introduces a novel model merging Self-Attention with Message Passing in GNNs, achieving a 15\% reduction in RMSE on the well known flow past a cylinder benchmark. Furthermore, a dynamic mesh pruning technique based on Self-Attention is proposed, that leads to a robust GNN-based multigrid approach, also reducing RMSE by 15\%. Additionally, a new self-supervised training method based on BERT is presented, resulting in a 25\% RMSE reduction. The paper includes an ablation study and outperforms state-of-the-art models on several challenging datasets, promising advancements similar to those recently achieved in natural language and image processing. Finally, the paper introduces a dataset with meshes larger than existing ones by at least an order of magnitude. Code and Datasets will be released at https://github.com/DonsetPG/multigrid-gnn.

References in corpus (7)

• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Masked Autoencoders Are Scalable Vision Learners
• U-net architectures for fast prediction of incompressible laminar flows
• MGAE: Masked Autoencoders for Self-Supervised Learning on Graphs
• Efficient Learning of Mesh-Based Physical Simulation with BSMS-GNN
• Simulating Continuum Mechanics with Multi-Scale Graph Neural Networks
• MG-GNN: Multigrid Graph Neural Networks for Learning Multilevel Domain Decomposition Methods