Optimizing the hybrid parallelization of BHAC

Abstract

We present our experience with the modernization on the GR-MHD code BHAC, aimed at improving its novel hybrid (MPI+OpenMP) parallelization scheme. In doing so, we showcase the use of performance profiling tools usable on x86 (Intel-based) architectures. Our performance characterization and threading analysis provided guidance in improving the concurrency and thus the efficiency of the OpenMP parallel regions. We assess scaling and communication patterns in order to identify and alleviate MPI bottlenecks, with both runtime switches and precise code interventions. The performance of optimized version of BHAC improved by $\sim 28\%$, making it viable for scaling on several hundreds of supercomputer nodes. We finally test whether porting such optimizations to different hardware is likewise beneficial on the new architecture by running on ARM A64FX vector nodes.

arXiv comment10 pages, 9 figures, 1 table; in review

References in corpus (14)

• GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral
• First M87 Event Horizon Telescope Results. I. The Shadow of the Supermassive Black Hole
• First M87 Event Horizon Telescope Results. V. Physical Origin of the Asymmetric Ring
• The Event Horizon General Relativistic Magnetohydrodynamic Code Comparison Project
• Magnetic Reconnection and Hot Spot Formation in Black Hole Accretion Disks
• Evidence for X-ray synchrotron emission from simultaneous mid-IR to X-ray observations of a strong Sgr A* flare
• How to tell an accreting boson star from a black hole
• Flares in the Galactic center I: orbiting flux tubes in Magnetically Arrested Black Hole Accretion Disks
• Plasmoid formation in global GRMHD simulations and AGN flares
• General relativistic resistive magnetohydrodynamics with robust primitive variable recovery for accretion disk simulations
• The Ecological Impact of High-performance Computing in Astrophysics
• 3D magnetised jet break-out from neutron-star binary merger ejecta: afterglow emission from the jet and the ejecta
• On the opening angle of magnetised jets from neutron-star mergers: the case of GRB170817A
• An astronomical institute's perspective on meeting the challenges of the climate crisis