A scalable elliptic solver with task-based parallelism for the SpECTRE numerical relativity code

Abstract

Elliptic partial differential equations must be solved numerically for many problems in numerical relativity, such as initial data for every simulation of merging black holes and neutron stars. Existing elliptic solvers can take multiple days to solve these problems at high resolution and when matter is involved, because they are either hard to parallelize or require a large amount of computational resources. Here we present a new solver for linear and nonlinear elliptic problems that is designed to scale with resolution and to parallelize on computing clusters. To achieve this we employ a discontinuous Galerkin discretization, an iterative multigrid-Schwarz preconditioned Newton-Krylov algorithm, and a task-based parallelism paradigm. To accelerate convergence of the elliptic solver we have developed novel subdomain-preconditioning techniques. We find that our multigrid-Schwarz preconditioned elliptic solves achieve iteration counts that are independent of resolution, and our task-based parallel programs scale over 200 million degrees of freedom to at least a few thousand cores. Our new code solves a classic initial data problem for binary black holes faster than the spectral code SpEC when distributed to only eight cores, and in a fraction of the time on more cores. It is publicly accessible in the next-generation SpECTRE numerical relativity code. Our results pave the way for highly parallel elliptic solves in numerical relativity and beyond.

25 pages, 20 figures, published version. Results are reproducible with the ancillary input files

References in corpus (12)

• GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral
• GWTC-2: Compact Binary Coalescences Observed by LIGO and Virgo During the First Half of the Third Observing Run
• Observation of gravitational waves from two neutron star-black hole coalescences
• Binary-black-hole initial data with nearly-extremal spins
• Initial data for black hole-neutron star binaries: a flexible, high-accuracy spectral method
• A new public code for initial data of unequal-mass, spinning compact-object binaries
• GRAthena++: puncture evolutions on vertex-centered oct-tree AMR
• Constructing Binary Neutron Star Initial Data with High Spins, High Compactness, and High Mass-Ratios
• HDGlab: An open-source implementation of the hybridisable discontinuous Galerkin method in MATLAB
• Initial data for black hole-neutron star binaries, with rotating stars
• Unified discontinuous Galerkin scheme for a large class of elliptic equations
• Numerically modeling Brownian thermal noise in amorphous and crystalline thin coatings

Cited by in corpus (10)

• The Science of the Einstein Telescope
• Elliptica: a new pseudo-spectral code for the construction of initial data
• NRPyElliptic: A Fast Hyperbolic Relaxation Elliptic Solver for Numerical Relativity, I: Conformally Flat, Binary Puncture Initial Data
• Unified discontinuous Galerkin scheme for a large class of elliptic equations
• Simulating binary black hole mergers using discontinuous Galerkin methods
• Quasistationary hair for binary black hole initial data in scalar Gauss-Bonnet gravity
• High-accuracy numerical models of Brownian thermal noise in thin mirror coatings
• Discontinuous Galerkin scheme for elliptic equations on extremely stretched grids
• Eccentricity as a signature of hierarchical subsolar-mass mergers in collapsar disks
• Parameter control for binary black hole initial data