GALÆXI: Solving complex compressible flows with high-order discontinuous Galerkin methods on accelerator-based systems

Abstract

This work presents GALAEXI as a novel, energy-efficient flow solver for the simulation of compressible flows on unstructured meshes leveraging the parallel computing power of modern Graphics Processing Units (GPUs). GALAEXI implements the high-order Discontinuous Galerkin Spectral Element Method (DGSEM) using shock capturing with a finite-volume subcell approach to ensure the stability of the high-order scheme near shocks. This work provides details on the general code design, the parallelization strategy, and the implementation approach for the compute kernels with a focus on the element local mappings between volume and surface data due to the unstructured mesh. GALAEXI exhibits excellent strong scaling properties up to 1024 GPUs if each GPU is assigned a minimum of one million degrees of freedom degrees of freedom. To verify its implementation, a convergence study is performed that recovers the theoretical order of convergence of the implemented numerical schemes. Moreover, the solver is validated using both the incompressible and compressible formulation of the Taylor-Green-Vortex at a Mach number of 0.1 and 1.25, respectively. A mesh convergence study shows that the results converge to the high-fidelity reference solution and that the results match the original CPU implementation. Finally, GALAEXI is applied to a large-scale wall-resolved large eddy simulation of a linear cascade of the NASA Rotor 37. Here, the supersonic region and shocks at the leading edge are captured accurately and robustly by the implemented shock-capturing approach. It is demonstrated that GALAEXI requires less than half of the energy to carry out this simulation in comparison to the reference CPU implementation. This renders GALAEXI as a potent tool for accurate and efficient simulations of compressible flows in the realm of exascale computing and the associated new HPC architectures.

arXiv comment19 pages, 12 figures, 3 tables. Accepted Manuscript. Code available at: https://github.com/flexi-framework/galaexi

References in corpus (13)

• MFEM: a modular finite element methods library
• A Posteriori Subcell Limiting of the Discontinuous Galerkin Finite Element Method for Hyperbolic Conservation Laws
• Split Form Nodal Discontinuous Galerkin Schemes with Summation-By-Parts Property for the Compressible Euler Equations
• The deal.II finite element library: design, features, and insights
• Nektar++: enhancing the capability and application of high-fidelity spectral/$hp$ element methods
• Viscous Shock Capturing in a Time-Explicit Discontinuous Galerkin Method
• A provably entropy stable subcell shock capturing approach for high order split form DG for the compressible Euler Equations
• Space-time adaptive ADER-DG schemes for dissipative flows: compressible Navier-Stokes and resistive MHD equations
• HORSES3D: a high-order discontinuous Galerkin solver for flow simulations and multi-physics applications
• Adaptive numerical simulations with Trixi.jl: A case study of Julia for scientific computing
• Subcell limiting strategies for discontinuous Galerkin spectral element methods
• A matrix-free high-order discontinuous Galerkin compressible Navier-Stokes solver: A performance comparison of compressible and incompressible formulations for turbulent incompressible flows
• Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning