GRaM-X: A new GPU-accelerated dynamical spacetime GRMHD code for Exascale computing with the Einstein Toolkit
arXiv:2210.17509 · doi:10.1088/1361-6382/acf2d9
Abstract
We present GRaM-X (General Relativistic accelerated Magnetohydrodynamics on AMReX), a new GPU-accelerated dynamical-spacetime general relativistic magnetohydrodynamics (GRMHD) code which extends the GRMHD capability of Einstein Toolkit to GPU-based exascale systems. GRaM-X supports 3D adaptive mesh refinement (AMR) on GPUs via a new AMR driver for the Einstein Toolkit called CarpetX which in turn leverages AMReX, an AMR library developed for use by the United States DOE's Exascale Computing Project (ECP). We use the Z4c formalism to evolve the equations of GR and the Valencia formulation to evolve the equations of GRMHD. GRaM-X supports both analytic as well as tabulated equations of state. We implement TVD and WENO reconstruction methods as well as the HLLE Riemann solver. We test the accuracy of the code using a range of tests on static spacetime, e.g. 1D MHD shocktubes, the 2D magnetic rotor and a cylindrical explosion, as well as on dynamical spacetimes, i.e. the oscillations of a 3D TOV star. We find excellent agreement with analytic results and results of other codes reported in literature. We also perform scaling tests and find that GRaM-X shows a weak scaling efficiency of on 2304 nodes (13824 NVIDIA V100 GPUs) with respect to single-node performance on OLCF's supercomputer Summit.
arXiv comment22 pages, 8 figures, to be submitted to Classical and Quantum Gravity
References in corpus (51)
- Modules for Experiments in Stellar Astrophysics (MESA)
- Modules for Experiments in Stellar Astrophysics (MESA): Giant Planets, Oscillations, Rotation, and Massive Stars
- On the Numerical Integration of Einstein's Field Equations
- HARM: A Numerical Scheme for General Relativistic Magnetohydrodynamics
- The Einstein Toolkit: A Community Computational Infrastructure for Relativistic Astrophysics
- Gauge conditions for long-term numerical black hole evolutions without excision
- The Athena++ Adaptive Mesh Refinement Framework: Design and Magnetohydrodynamic Solvers
- Magnetorotationally driven Supernovae as the origin of early galaxy -process elements?
- Calibration of Moving Puncture Simulations
- A New Mechanism for Core-Collapse Supernova Explosions
- ECHO: an Eulerian Conservative High Order scheme for general relativistic magnetohydrodynamics and magnetodynamics
- A large scale dynamo and magnetoturbulence in rapidly rotating core-collapse supernovae
- Constraint violation in free evolution schemes: comparing BSSNOK with a conformal decomposition of Z4
- An efficient shock-capturing central-type scheme for multidimensional relativistic flows. II. Magnetohydrodynamics
- Efficient magnetic-field amplification due to the Kelvin-Helmholtz instability in binary neutron star mergers
- Binary neutron star mergers: a jet engine for short gamma-ray bursts
- Turduckening black holes: an analytical and computational study
- Compact binary evolutions with the Z4c formulation
- Three-dimensional general relativistic hydrodynamics II: long-term dynamics of single relativistic stars
- GENESIS: A high-resolution code for 3D relativistic hydrodynamics
- High-resolution magnetohydrodynamics simulation of black hole-neutron star merger: Mass ejection and short gamma-ray burst
- General relativistic magnetohydrodynamic simulations of binary neutron star mergers forming a long-lived neutron star
- WhiskyMHD: a new numerical code for general relativistic magnetohydrodynamics
- Numerical 3+1 general relativistic magnetohydrodynamics: a local characteristic approach
- Semi-global simulations of the magneto-rotational instability in core collapse supernovae
- Towards a Stable Numerical Evolution of Strongly Gravitating Systems in General Relativity: The Conformal Treatments
- R-process Nucleosynthesis from Three-Dimensional Magnetorotational Core-Collapse Supernovae
- General relativistic simulations of compact binary mergers as engines of short gamma-ray bursts
- GRHydro: A new open source general-relativistic magnetohydrodynamics code for the Einstein Toolkit
- IllinoisGRMHD: An Open-Source, User-Friendly GRMHD Code for Dynamical Spacetimes
- First 100 ms of a long-lived magnetized neutron star formed in a binary neutron star merger
- The Exact Solution of the Riemann Problem in Relativistic MHD
- A magnetar engine for short GRBs and kilonovae
- Relativistic magnetohydrodynamics in dynamical spacetimes: A new AMR implementation
- Magnetorotational core collapse of possible GRB progenitors. I. Explosion mechanisms
- SpECTRE: A Task-based Discontinuous Galerkin Code for Relativistic Astrophysics
- A Second Order Godunov Method for Multidimensional Relativistic Magnetohydrodynamics
- Relativistic MHD with Adaptive Mesh Refinement
- Implications of the search for optical counterparts during the first six months of the Advanced LIGO's and Advanced Virgo's third observing run: possible limits on the ejecta mass and binary properties
- -Process Nucleosynthesis from Three-dimensional Jet-driven Core-Collapse Supernovae with Magnetic Misalignments
- Collimated outflows from long-lived binary neutron star merger remnants
- A new general relativistic magnetohydrodynamics code for dynamical spacetimes
- Modeling magnetized neutron stars using resistive MHD
- Constraint preserving boundary conditions for the Z4c formulation of general relativity
- The Challenges Ahead for Multimessenger Analyses of Gravitational Waves and Kilonova: a Case Study on GW190425
- Recovery schemes for primitive variables in general-relativistic magnetohydrodynamics
- Quasi-radial modes of rotating stars in general relativity
- Implementation of advanced Riemann solvers in a neutrino-radiation magnetohydrodynamics code in numerical relativity and its application to a binary neutron star merger
- Tabulated equation of state for supernova matter including full nuclear ensemble
- The Merger of Small and Large Black Holes
- Three Dimensional Numerical Relativity with a Hyperbolic Formulation
Cited by in corpus (6)
- The new discontinuous Galerkin methods based numerical relativity program Nmesh
- Performance-Portable Binary Neutron Star Mergers with AthenaK
- AsterX: a new open-source GPU-accelerated GRMHD code for dynamical spacetimes
- Gravitational waves from magnetorotational core-collapse supernovae using 3D GRMHD simulations: effect of rotation and magnetic fields
- Neutron star evolution by combining discontinuous Galerkin and finite volume methods
- A Black Hole is Born: 3D GRMHD Simulation of Black Hole Formation from Core-Collapse