Quokka: A code for two-moment AMR radiation hydrodynamics on GPUs

Abstract

We present Quokka, a new subcycling-in-time, block-structured adaptive mesh refinement (AMR) radiation hydrodynamics code optimised for graphics processing units (GPUs). Quokka solves the equations of hydrodynamics with the piecewise parabolic method (PPM) in a method-of-lines formulation, and handles radiative transfer via the variable Eddington tensor (VET) radiation moment equations with a local closure. We use the AMReX library to handle the adaptive mesh management. In order to maximise GPU performance, we combine explicit-in-time evolution of the radiation moment equations with the reduced speed-of-light approximation. We show results for a wide range of test problems for hydrodynamics, radiation, and coupled radiation hydrodynamics. On uniform grids in 3D on a single GPU, our code achieves > 250 million hydrodynamic updates per second and almost 40 million radiation hydrodynamic updates per second. For radiation hydrodynamics problems on uniform grids in 3D, our code scales from 4 GPUs to 256 GPUs with an efficiency of 76 per cent. The code is publicly released under an open-source license on GitHub.

Revised version re-submitted to MNRAS, added new performance measurements with A100 GPUs

References in corpus (13)

• Array Programming with NumPy
• Athena: A New Code for Astrophysical MHD
• The Athena++ Adaptive Mesh Refinement Framework: Design and Magnetohydrodynamic Solvers
• The RAGE radiation-hydrodynamic code
• The Effects of Radiative Transfer on Low-Mass Star Formation
• Equations and Algorithms for Mixed Frame Flux-Limited Diffusion Radiation Hydrodynamics
• Hydrodynamical Coupling of Mass and Momentum in Multiphase Galactic Winds
• Magnetohydronamic Evolution of HII Regions in Molecular Clouds: Simulation Methodology, Tests, and Uniform Media
• Cholla : A New Massively-Parallel Hydrodynamics Code For Astrophysical Simulation
• Radiation Feedback in ULIRGS: Are Photons Movers and Shakers?
• Hybrid Adaptive Ray-Moment Method (HARM${}^2$): A Highly Parallel Method for Radiation Hydrodynamics on Adaptive Grids
• A multigroup diffusion solver using pseudo transient continuation for a radiation-hydrodynamic code with patch-based AMR
• A high-performance and portable asymptotic preserving radiation hydrodynamics code with the M1 model

Cited by in corpus (12)

• Why are thermally- and cosmic ray-driven galactic winds fundamentally different?
• Radiation transport methods in star formation simulations
• An Asymptotically-Correct Implicit-Explicit Time Integration Scheme for Finite Volume Radiation-Hydrodynamics
• High-order Discontinuous Galerkin hydrodynamics with sub-cell shock capturing on GPUs
• Radiation magnetohydrodynamics simulations of Population III star formation during the Epoch of Reionization
• Quokka-based understanding of outflows (QED) -- II. X-ray metallicity gradients as a signature of galactic wind metal loading
• A novel numerical method for mixed-frame multigroup radiation-hydrodynamics with GPU acceleration implemented in the QUOKKA code
• Neural-Network Chemical Emulator for First-Star Formation: Robust Iterative Predictions over a Wide Density Range
• QED V: Variations in metal loading of galactic winds with element nucleosynthetic origin
• AMReX-Astrophysics Microphysics: A set of microphysics routines for astrophysical simulation codes based on the AMReX library
• Quokka-based understanding of outflows (QED) -- IV. Limitations of H$\alpha$ as an outflow diagnostic
• HARD: A Performance Portable Radiation Hydrodynamics Code based on FleCSI Framework