Optimizing the hybrid parallelization of BHAC

Abstract

We present our experience with the modernization on the GR-MHD code BHAC, aimed at improving its novel hybrid (MPI+OpenMP) parallelization scheme. In doing so, we showcase the use of performance profiling tools usable on x86 (Intel-based) architectures. Our performance characterization and threading analysis provided guidance in improving the concurrency and thus the efficiency of the OpenMP parallel regions. We assess scaling and communication patterns in order to identify and alleviate MPI bottlenecks, with both runtime switches and precise code interventions. The performance of optimized version of BHAC improved by $\sim 28\%$, making it viable for scaling on several hundreds of supercomputer nodes. We finally test whether porting such optimizations to different hardware is likewise beneficial on the new architecture by running on ARM A64FX vector nodes.

10 pages, 9 figures, 1 table; in review

References in corpus (5)

• GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral
• First M87 Event Horizon Telescope Results. I. The Shadow of the Supermassive Black Hole
• First M87 Event Horizon Telescope Results. V. Physical Origin of the Asymmetric Ring
• The Ecological Impact of High-performance Computing in Astrophysics
• An astronomical institute's perspective on meeting the challenges of the climate crisis