Cosmological neutrino simulations at extreme scale

Abstract

Constraining neutrino mass remains an elusive challenge in modern physics. Precision measurements are expected from several upcoming cosmological probes of large-scale structure. Achieving this goal relies on an equal level of precision from theoretical predictions of neutrino clustering. Numerical simulations of the non-linear evolution of cold dark matter and neutrinos play a pivotal role in this process. We incorporate neutrinos into the cosmological N-body code CUBEP3M and discuss the challenges associated with pushing to the extreme scales demanded by the neutrino problem. We highlight code optimizations made to exploit modern high performance computing architectures and present a novel method of data compression that reduces the phase-space particle footprint from 24 bytes in single precision to roughly 9 bytes. We scale the neutrino problem to the Tianhe-2 supercomputer and provide details of our production run, named TianNu, which uses 86% of the machine (13,824 compute nodes). With a total of 2.97 trillion particles, TianNu is currently the world's largest cosmological N-body simulation and improves upon previous neutrino simulations by two orders of magnitude in scale. We finish with a discussion of the unanticipated computational challenges that were encountered during the TianNu runtime.

Accepted for publication in Research in Astronomy and Astrophysics

References in corpus (7)

• Massive neutrinos and cosmology
• Neutrino masses and mixings: Status of known and unknown $3\nu$ parameters
• P3DFFT: a framework for parallel computations of Fourier transforms in three dimensions
• The Effect of Thermal Neutrino Motion on the Non-linear Cosmological Matter Power Spectrum
• Precision reconstruction of the dark matter-neutrino relative velocity from N-body simulations
• Differential Neutrino Condensation onto Cosmic Structure
• Measurement of the Cold Dark Matter-Neutrino Dipole in the TianNu Simulation

Cited by in corpus (48)

• AbacusSummit: A Massive Set of High-Accuracy, High-Resolution $N$-Body Simulations
• Large-scale dark matter simulations
• Early Structure Formation in $\Lambda$PBH Cosmologies
• Constraining $M_{\nu }$ with the Bispectrum I: Breaking Parameter Degeneracies
• MassiveNuS: Cosmological Massive Neutrino Simulations
• Constraining $M_{\nu }$ with the Bispectrum II: The Total Information Content of the Galaxy Bispectrum
• The imprint of neutrinos on clustering in redshift-space
• Detecting Neutrino Mass by Combining Matter Clustering, Halos, and Voids
• Relativistic N-body simulations with massive neutrinos
• An optimal nonlinear method for simulating relic neutrinos
• An Efficient and Accurate Hybrid Method for Simulating Non-Linear Neutrino Structure
• Scale-dependent bias and bispectrum in neutrino separate universe simulations
• DEMNUni: Massive neutrinos and the bispectrum of large scale structures
• Reducing Noise in Cosmological N-body Simulations with Neutrinos
• $\nu \mathrm{CO}N\mathrm{CEPT}$: Cosmological neutrino simulations from the non-linear Boltzmann hierarchy
• Gravitational clustering of cosmic relic neutrinos in the Milky Way
• First detection of scale-dependent linear halo bias in $N$-body simulations with massive neutrinos
• Accurate fitting functions for peculiar velocity spectra in standard and massive-neutrino cosmologies
• Fully relativistic treatment of light neutrinos in $N$-body simulations
• A fast particle-mesh simulation of non-linear cosmological structure formation with massive neutrinos
• Beware of Fake $\nu$s: The Effect of Massive Neutrinos on the Nonlinear Evolution of Cosmic Structure
• On the road to percent accuracy III: non-linear reaction of the matter power spectrum to massive neutrinos
• Euclid: Modelling massive neutrinos in cosmology -- a code comparison
• Improved Mixed Dark Matter Halo Model for Ultralight Axions
• CUBE: An Information-optimized parallel Cosmological $N$-body Algorithm
• Fast simulations of cosmic large-scale structure with massive neutrinos
• Parity-odd Neutrino Torque Detection
• Neutrino effects on the morphology of cosmic large-scale structure
• Probing massive neutrinos with the Minkowski functionals of large-scale structure
• Massive neutrinos and degeneracies in Lyman-alpha forest simulations
• Geodesic motion and phase-space evolution of massive neutrinos
• The Sejong Suite: Cosmological Hydrodynamical Simulations with Massive Neutrinos, Dark Radiation, and Warm Dark Matter
• Impact of Massive Neutrinos and Dark Radiation on the High-redshift Cosmic Web. I. Lyman-Alpha Forest Observables
• Momentum space sampling of neutrinos in $N$-body simulations
• Measuring dark matter-neutrino relative velocity on cosmological scales
• Neutrino winds on the sky
• Relativistic Corrections to the Growth of Structure in Modified Gravity
• PHoToNs--A Parallel Heterogeneous & Threads oriented code for cosmological N-body simulation
• CUBE -- Towards an Optimal Scaling of Cosmological N-body Simulations
• Improving initialization and evolution accuracy of cosmological neutrino simulations
• Neutrinos in N-body simulations
• Perturbation-theory informed integrators for cosmological simulations
• Effect of Massive Neutrinos on the Position of Cold Dark Matter Halo: Revealed via Delaunay Triangulation Void
• Measuring neutrino mass and asymmetry with matter pairwise velocities
• Cosmological neutrino N-body simulations of dark matter halo
• Effect of Neutrinos on Angular Momentum of Dark Matter Halo
• Slow neutrinos: non-linearity and momentum-space emulation
• A 400 Trillion-Grid Vlasov Simulation on Fugaku Supercomputer: Large-Scale Distribution of Cosmic Relic Neutrinos in a Six-dimensional Phase Space