SWIFT: A modern highly-parallel gravity and smoothed particle hydrodynamics solver for astrophysical and cosmological applications

Abstract

Numerical simulations have become one of the key tools used by theorists in all the fields of astrophysics and cosmology. The development of modern tools that target the largest existing computing systems and exploit state-of-the-art numerical methods and algorithms is thus crucial. In this paper, we introduce the fully open-source highly-parallel, versatile, and modular coupled hydrodynamics, gravity, cosmology, and galaxy-formation code SWIFT. The software package exploits hybrid shared- and distributed-memory task-based parallelism, asynchronous communications, and domain-decomposition algorithms based on balancing the workload, rather than the data, to efficiently exploit modern high-performance computing cluster architectures. Gravity is solved for using a fast-multipole-method, optionally coupled to a particle mesh solver in Fourier space to handle periodic volumes. For gas evolution, multiple modern flavours of Smoothed Particle Hydrodynamics are implemented. SWIFT also evolves neutrinos using a state-of-the-art particle-based method. Two complementary networks of sub-grid models for galaxy formation as well as extensions to simulate planetary physics are also released as part of the code. An extensive set of output options, including snapshots, light-cones, power spectra, and a coupling to structure finders are also included. We describe the overall code architecture, summarise the consistency and accuracy tests that were performed, and demonstrate the excellent weak-scaling performance of the code using a representative cosmological hydrodynamical problem with $\approx$$300$ billion particles. The code is released to the community alongside extensive documentation for both users and developers, a large selection of example test problems, and a suite of tools to aid in the analysis of large simulations run with SWIFT.

43 pages, 20 figures, accepted for publication in MNRAS. Code, documentation, and examples available at www.swiftsim.com

References in corpus (155)

• A Universal Density Profile from Hierarchical Clustering
• The cosmological simulation code GADGET-2
• HEALPix -- a Framework for High Resolution Discretization, and Fast Analysis of Data Distributed on the Sphere
• The Global Schmidt Law in Star Forming Galaxies
• Simulating the joint evolution of quasars, galaxies and their large-scale distribution
• The EAGLE project: Simulating the evolution and assembly of galaxies and their environments
• E pur si muove: Galiliean-invariant cosmological hydrodynamical simulations on a moving mesh
• Advection-Dominated Accretion: A Self-Similar Solution
• Cosmological SPH simulations: A hybrid multi-phase model for star formation
• Modeling feedback from stars and black holes in galaxy mergers
• Cosmological Hydrodynamics with Adaptive Mesh Refinement: a new high resolution code called RAMSES
• GADGET: A code for collisionless and gasdynamical cosmological simulations
• The EAGLE simulations of galaxy formation: calibration of subgrid physics and model variations
• Physical Models of Galaxy Formation in a Cosmological Framework
• A Multi-Code Analysis Toolkit for Astrophysical Simulation Data
• Radiative transfer in a clumpy universe: IV. New synthesis models of the cosmic UV/X-ray background
• A Cosmology Calculator for the World Wide Web
• GIZMO: A New Class of Accurate, Mesh-Free Hydrodynamic Simulation Methods
• Massive neutrinos and cosmology
• Common Envelope Evolution: Where we stand and how we can move forward
• The Inner Structure of LambdaCDM Halos I: A Numerical Convergence Study
• The Large Scale Bias of Dark Matter Halos: Numerical Calibration and Model Tests
• Cosmological Simulations with TreeSPH
• The effect of photo-ionization on the cooling rates of enriched, astrophysical plasmas
• Cosmological SPH simulations: The entropy equation
• Enzo: An Adaptive Mesh Refinement Code for Astrophysics
• Cosmological simulations of the growth of supermassive black holes and feedback from active galactic nuclei: method and tests
• Relic neutrino decoupling including flavour oscillations
• Improving convergence in smoothed particle hydrodynamics simulations without pairing instability
• Star Formation and Feedback in Smoothed Particle Hydrodynamic Simulations--I. Isolated Galaxies
• Smoothed Particle Hydrodynamics and Magnetohydrodynamics
• Theoretical Challenges in Galaxy Formation
• Gasoline: An adaptable implementation of TreeSPH
• Chemical enrichment in cosmological, smoothed particle hydrodynamics simulations
• Adaptive Refinement Tree - a new high-resolution N-body code for cosmological simulations
• On the relation between the Schmidt and Kennicutt-Schmidt star formation laws and its implications for numerical simulations
• Fundamental differences between SPH and grid methods
• Star formation thresholds and galaxy edges: why and where
• The Athena++ Adaptive Mesh Refinement Framework: Design and Magnetohydrodynamic Solvers
• Evolution of structure in cold dark matter universes
• Simulating galactic outflows with thermal supernova feedback
• The PLUTO Code for Adaptive Mesh Computations in Astrophysical Fluid Dynamics
• Self-Regulated Star Formation in Galaxies via Momentum Input from Massive Stars
• Phantom: A smoothed particle hydrodynamics and magnetohydrodynamics code for astrophysics
• A General Class of Lagrangian Smoothed Particle Hydrodynamics Methods and Implications for Fluid Mixing Problems
• Simulating cosmic structure formation with the GADGET-4 code
• Metallicity of the intergalactic medium using pixel statistics. II. The distribution of metals as traced by C IV
• Smoothed Particle Hydrodynamics in Astrophysics
• The Cosmic Linear Anisotropy Solving System (CLASS) IV: Efficient implementation of non-cold relics
• Modelling discontinuities and Kelvin-Helmholtz instabilities in SPH
• Chemical enrichment of galaxy clusters from hydrodynamical simulations
• Grackle: a Chemistry and Cooling Library for Astrophysics
• An energy-conserving formalism for adaptive gravitational force softening in SPH and N-body codes
• Inviscid SPH
• Cosmic Structure Growth and Dark Energy
• Molecular Hydrogen and Global Star Formation Relations in Galaxies
• Inferring the dark matter power spectrum from the Lyman-alpha forest in high-resolution QSO absorption spectra
• The impact of angular momentum on black hole accretion rates in simulations of galaxy formation
• Hydra: An Adaptive--Mesh Implementation of PPPM--SPH
• Nyx: A Massively Parallel AMR Code for Computational Cosmology
• Gasoline2: A Modern SPH Code
• A Hierarchical O(N) Force Calculation Algorithm
• The FLAMINGO project: cosmological hydrodynamical simulations for large-scale structure and galaxy cluster surveys
• Colours and luminosities of z=0.1 simulated galaxies in the EAGLE simulations
• Galaxy And Mass Assembly (GAMA): Data Release 4 and the z < 0.1 total and z < 0.08 morphological galaxy stellar mass functions
• A Very Fast and Momentum-Conserving Tree Code
• A Cosmic UV/X-ray Background Model Update
• HACC: Simulating Sky Surveys on State-of-the-Art Supercomputing Architectures
• The EAGLE simulations of galaxy formation: the importance of the hydrodynamics scheme
• The effects of feedback on the morphology of galaxy discs
• A Necessary Condition for Individual Time-Steps in SPH Simulations
• Structure Finding in Cosmological Simulations: The State of Affairs
• Implementation of feedback in SPH: towards concordance of methods
• Towards optimal softening in 3D N-body codes: I. Minimizing the force error
• Large-scale dark matter simulations
• Matter power spectrum and the challenge of percent accuracy
• An efficient implementation of massive neutrinos in non-linear structure formation simulations
• High Performance P3M N-body code: CUBEP3M
• gevolution: a cosmological N-body code based on General Relativity
• N-body simulations of gravitational dynamics
• The AGORA High-Resolution Galaxy Simulations Comparison Project. II: Isolated Disk Test
• Hydrodynamical simulations of the galaxy population: enduring successes and outstanding challenges
• Variable Smoothing Lengths and Energy Conservation in Smoothed Particle Hydrodynamics
• A New Parallel N-body Gravity Solver: TPM
• The Cosmic Code Comparison Project
• The $\text{Abacus}$ Cosmological $N$-body Code
• Pushing back the limits: detailed properties of dwarf galaxies in a LCDM universe
• Reionization simulations powered by GPUs I: the structure of the Ultraviolet radiation field
• Parallel TreeSPH
• Radiative cooling rates, ion fractions, molecule abundances and line emissivities including self-shielding and both local and metagalactic radiation fields
• The impact of local stellar radiation on the HI column density distribution
• TreePM: A code for Cosmological N-Body Simulations
• Astrophysical Weighted Particle Magnetohydrodynamics
• A High-Fidelity Realization of the Euclid Code Comparison $N$-body Simulation with Abacus
• Accurate initial conditions for cosmological N-body simulations: Minimizing truncation and discreteness errors
• Accurate estimators of power spectra in N-body simulations
• Hunting for Galaxies and Halos in simulations with VELOCIraptor
• CRKSPH - A Conservative Reproducing Kernel Smoothed Particle Hydrodynamics Scheme
• Numerical convergence of simulations of galaxy formation: the abundance and internal structure of cold dark matter haloes
• Initial Conditions for Accurate N-Body Simulations of Massive Neutrino Cosmologies
• SEREN - A new SPH code for star and planet formation simulations
• The Dynamical and Chemical Evolution of Dwarf Spheroidal Galaxies with GEAR
• Non-equilibrium chemistry and cooling in the diffuse interstellar medium - I. Optically thin regime
• Non-equilibrium chemistry and cooling in the diffuse interstellar medium - II. Shielded gas
• A new equation of state for dense hydrogen-helium mixtures II: taking into account hydrogen-helium interactions
• nIFTy galaxy cluster simulations I: dark matter & non-radiative models
• Sphenix: Smoothed Particle Hydrodynamics for the next generation of galaxy formation simulations
• The Ecological Impact of High-performance Computing in Astrophysics
• Chemical mixing in smoothed particle hydrodynamics simulations
• The Lyman-$\alpha$ forest as a diagnostic of the nature of the dark matter
• An optimal nonlinear method for simulating relic neutrinos
• Higher-order initial conditions for mixed baryon-CDM simulations
• Peaks above the Maxwellian Sea: A New Approach to Finding Substructure in N-Body Haloes
• Absorption signatures of warm-hot gas at low redshift: OVI
• GRAPESPH with Fully Periodic Boundary Conditions: Fragmentation of Molecular Clouds
• The importance of black hole repositioning for galaxy formation simulations
• Spin-driven jet feedback in idealised simulations of galaxy groups and clusters
• Damn You, Little h! (or, Real-World Applications Of The Hubble Constant Using Observed And Simulated Data)
• 2HOT: An Improved Parallel Hashed Oct-Tree N-Body Algorithm for Cosmological Simulation
• SWIFT: Using task-based parallelism, fully asynchronous communication, and graph partition-based domain decomposition for strong scaling on more than 100,000 cores
• Computational issues in chemo-dynamical modelling of the formation and evolution of galaxies
• The Shock Physics of Giant Impacts: Key Requirements for the Equations of State
• Planetary Giant Impacts: Convergence of High-Resolution Simulations using Efficient Spherical Initial Conditions and SWIFT
• Numerical Simulations of the Lyman-alpha forest - A comparison of Gadget-2 and Enzo
• SIBELIUS-DARK: a galaxy catalogue of the Local Volume from a constrained realisation simulation
• Satellite dwarf galaxies: Stripped but not quenched
• The AGORA High-resolution Galaxy Simulations Comparison Project. III: Cosmological zoom-in simulation of a Milky Way-mass halo
• A thermal-kinetic subgrid model for supernova feedback in simulations of galaxy formation
• Performance Characteristics of TreePM codes
• The Lagrangian hydrodynamics code MAGMA2
• A new paradigm for reproducing and analyzing N-body simulations of planetary systems
• Immediate origin of the Moon as a post-impact satellite
• Constraining the primordial magnetic field with dwarf galaxy simulations
• unyt: Handle, manipulate, and convert data with units in Python
• The DESI $N$-body Simulation Project I: Testing the Robustness of Simulations for the DESI Dark Time Survey
• How much metals did the first stars provide to the ultra-faint dwarfs?
• Baryons, Dark Matter, and the Jeans Mass in Simulations of Cosmological Structure Formation
• The impact of cored density profiles on the observable quantities of dwarf spheroidal galaxies
• The effect of pre-impact spin on the Moon-forming collision
• A Vector Potential implementation for Smoothed Particle Magnetohydrodynamics
• The moving mesh code Shadowfax
• EAGLE-like simulation models do not solve the entropy core problem in groups and clusters of galaxies
• Smoothed Particle Radiation Hydrodynamics: Two-Moment method with Local Eddington Tensor Closure
• Geodesic motion and phase-space evolution of massive neutrinos
• Active galactic nuclei jets simulated with smoothed particle hydrodynamics
• Novel Conservative Methods for Adaptive Force Softening in Collisionless and Multi-Species N-Body Simulations
• Inconsistencies arising from the coupling of galaxy formation sub-grid models to Pressure-Smoothed Particle Hydrodynamics
• Mesh-free hydrodynamics in PKDGRAV3 for galaxy formation simulations
• Sensitivity of non-radiative cloud-wind interactions to the hydrodynamics solver
• An Efficient SIMD Implementation of Pseudo-Verlet Lists for Neighbour Interactions in Particle-Based Codes
• Tree-less 3d Friends-of-Friends using Spatial Hashing
• A Hybrid MPI+Threads Approach to Particle Group Finding Using Union-Find
• Continuous Simulation Data Stream: A dynamical timescale-dependent output scheme for simulations
• GEAR-RT: Towards Exa-Scale Moment Based Radiative Transfer For Cosmological Simulations Using Task-Based Parallelism And Dynamic Sub-Cycling with SWIFT
• A simple and efficient solver for self-gravity in the DISPATCH astrophysical simulation framework

Cited by in corpus (55)

• The CosmoVerse White Paper: Addressing observational tensions in cosmology with systematics and fundamental physics
• A unified model for the clustering of quasars and galaxies at $z\approx 6$
• The FLAMINGO project: Baryon effects on the matter power spectrum
• SOAP: A Python Package for Calculating the Properties of Galaxies and Halos Formed in Cosmological Simulations
• Introducing the AIDA-TNG project: galaxy formation in alternative dark matter models
• Numerical challenges for energy conservation in N-body simulations of collapsing self-interacting dark matter halos
• Quantum algorithms for scientific computing
• On the accuracy of dark matter halo merger trees and the consequences for semi-analytic models of galaxy formation
• Constrained cosmological simulations of the Local Group using Bayesian hierarchical field-level inference
• Origin of Mars's moons by disruptive partial capture of an asteroid
• Two per cent measurement of $H_0$ from Cepheids alone
• Modelling the evolution and influence of dust in cosmological simulations that include the cold phase of the interstellar medium
• Hybrid-chimes: A model for radiative cooling and the abundances of ions and molecules in simulations of galaxy formation
• Cosmological feedback from a halo assembly perspective
• Learning the Universe: Cosmological and Astrophysical Parameter Inference with Galaxy Luminosity Functions and Colours
• Validating the clustering predictions of empirical models with the FLAMINGO simulations
• REMIX SPH -- improving mixing in smoothed particle hydrodynamics simulations using a generalised, material-independent approach
• Field-level simulation-based inference with galaxy catalogs: the impact of systematic effects
• BullFrog: Multi-step perturbation theory as a time integrator for cosmological simulations
• Non-explosive pre-supernova feedback in the COLIBRE model of galaxy formation
• Map-level baryonification: unified treatment of weak lensing two-point and higher-order statistics
• COLIBRE: calibrating subgrid feedback in cosmological simulations that include a cold gas phase
• Dark Matter-Induced Baryonic Feedback in Galaxies
• The FLAMINGO project: cosmology with the redshift dependence of weak gravitational lensing peaks
• Load Balancing For High Performance Computing Using Quantum Annealing
• Modeling submillimeter galaxies in cosmological simulations: Contribution to the cosmic star formation density and predictions for future surveys
• Tetrahedral grids in Monte Carlo radiative transfer
• syren-baryon: Analytic emulators for the impact of baryons on the matter power spectrum
• Exploring the catastrophic regime: thermodynamics and disintegration in head-on planetary collisions
• A 1% accurate method to include baryonic effects in galaxy-galaxy lensing models
• Cosmological Zoom-In Simulations of Milky Way Host Mass Dark Matter Halos with a Blue-Tilted Primordial Power Spectrum
• SWIFTGalaxy: a Python package to work with particle groups from SWIFT simulations
• Core to Cosmic Edge: SIMBA-C's New Take on Abundance Profiles in the Intragroup Medium at z = 0
• DARKSKIES: A suite of super-sampled zoom-in simulations of galaxy clusters with self-interacting dark matter
• Weak lensing constraints on the stellar-to-halo mass relation of galaxy groups with simulation-informed scatter
• Dynamical disequilibrium in dwarf galaxies: rethinking gas dynamics, rotation curves, and dark matter inference
• Annotation-guided AoS-to-SoA conversions and GPU offloading with data views in C++
• No dilute core produced in simulations of giant impacts on to Jupiter
• Kennicutt-Schmidt relation of galaxies over 13 billion years in the COLIBRE hydrodynamical simulations
• The role of gas stripping in the quenching of satellite galaxies using SHARK v2.0
• The FLAMINGO simulations data release
• Gravitational potential drives the concentration dependence of the stellar mass-halo mass relation
• Compiler support for semi-manual AoS-to-SoA conversions with data views
• Variability in Cosmological Hydrodynamical Simulations: how Stochastic Processes, Numerical Effects, and Reproducibility Limits impact Predictability
• Three-point intrinsic alignments of galaxies and haloes in the FLAMINGO simulations
• Kinematic dynamos and resolution limits for Smoothed Particle Magnetohydrodynamics
• Excess of substructure due to primordial black holes
• The tidal evolution of satellite galaxies in cosmological simulations: insights from COLIBRE
• Ray-traced weak lensing convergence in screened modified gravity theories
• The Manticore Project II: Bayesian digital twins of cosmic structure across the SDSS and BOSS volumes
• Ginnungagap -- a massively parallel cosmological initial conditions generator
• On the consistency of jet feedback modelling across different astrophysics hydrodynamical codes
• Slow neutrinos: non-linearity and momentum-space emulation
• Universal Fitting Formulae for the Peak Concentration of Dark Matter Halos
• Entropy plateaus can emerge from gas replacement at a characteristic halo mass in simulated groups and clusters of galaxies