HORSES3D-GPU: A high-order discontinuous Galerkin solver for multi-GPU systems
arXiv:2607.26674
The paper describes the GPU-accelerated version of the open-source high-order discontinuous Galerkin CFD solver HORSES3D, showing its performance and scalability on multi‑GPU systems for large turbulent flow simulations.
Abstract
We present the GPU acceleration and large-scale performance assessment of HORSES3D, an open-source high-order discontinuous Galerkin solver for computational fluid dynamics. The solver is ported to NVIDIA GPU architectures using OpenACC directives, preserving the original Fortran code structure while enabling GPU-resident execution of the main computational kernels. The implementation exploits the element-local structure of discontinuous Galerkin spectral element methods by mapping element-level loops to GPU gangs and nodal operations to vector-level parallelism. The GPU version is verified using the method of manufactured solutions and validated on canonical turbulent-flow benchmarks. Its performance is assessed on the MareNostrum 5 accelerated partition using NVIDIA H100 GPUs. Taylor-Green vortex benchmarks show that solver efficiency improves with polynomial order and that near-ideal strong and weak scaling is obtained when the workload exceeds approximately 16,000 to 20,000 elements per GPU. The solver is further evaluated on the High-Lift Common Research Model wing-body configuration, which involves a complex geometry, realistic boundary conditions, and unstructured meshes with up to 20.8 million hexahedral elements. Simulations with polynomial orders up to reach approximately degrees of freedom and scale efficiently to 2,048 GPUs. The results demonstrate that HORSES3D preserves its performance characteristics for industrially relevant configurations and can exploit modern GPU-based supercomputers for billion-degree-of-freedom high-order CFD simulations.