A 400 Trillion-Grid Vlasov Simulation on Fugaku Supercomputer: Large-Scale Distribution of Cosmic Relic Neutrinos in a Six-dimensional Phase Space

Abstract

We report a Vlasov simulation of cosmic relic neutrinos combined with N-body simulation of cold dark matter in the context of large-scale structure formation in the Universe performed on Fugaku supercomputer. Gravitational dynamics of the neutrinos is followed, for the first time, by directly integrating the Vlasov equation in a six-dimensional phase space. Our largest simulation combines the Vlasov simulation on 400 trillion grids and 330 billion-body calculations in a self-consistent manner, and reproduces accurately the nonlinear dynamics of neutrinos in the Universe. The novel high-order Vlasov solver is optimized by combining an array of state-of-the-art numerical schemes and fully utilizing the SIMD instructions on the A64FX processors. Time-To-Solution of our simulation is an order of magnitude shorter than the largest N-body simulations. The performance scales excellently with up to 147,456 nodes (7 million CPU cores) on Fugaku; the weak and strong scaling efficiencies are 82% - 96% and 82% - 93%, respectively.

arXiv comment11 pages, 8 figures. 2021 ACM Gordon-Bell Prize Finalist. A simulation movie is available at https://youtu.be/p2fHt8f7chQ

References in corpus (15)

• Planck 2015 results. XIII. Cosmological parameters
• Evidence for oscillation of atmospheric neutrinos
• Massive Neutrinos and the Non-linear Matter Power Spectrum
• TreePM: A code for Cosmological N-Body Simulations
• GRAPE-5: A Special-Purpose Computer for N-body Simulation
• GOTPM: A Parallel Hybrid Particle-Mesh Treecode
• Cosmological neutrino simulations at extreme scale
• Phantom-GRAPE: numerical software library to accelerate collisionless $N$-body simulation with SIMD instruction set on x86 architecture
• Performance Tuning of N-Body Codes on Modern Microprocessors: I. Direct Integration with a Hermite Scheme on x86_64 Architecture
• Precision reconstruction of the dark matter-neutrino relative velocity from N-body simulations
• Reducing Noise in Cosmological N-body Simulations with Neutrinos
• Direct Integration of the Collisionless Boltzmann Equation in Six-dimensional Phase Space: Self-gravitating Systems
• Differential Neutrino Condensation onto Cosmic Structure
• Measurement of the Cold Dark Matter-Neutrino Dipole in the TianNu Simulation
• Multidimensional VlasovPoisson Simulations with High-order Monotonicity- and Positivity-preserving Schemes