CUBE -- Towards an Optimal Scaling of Cosmological N-body Simulations

Abstract

N-body simulations are essential tools in physical cosmology to understand the large-scale structure (LSS) formation of the Universe. Large-scale simulations with high resolution are important for exploring the substructure of universe and for determining fundamental physical parameters like neutrino mass. However, traditional particle-mesh (PM) based algorithms use considerable amounts of memory, which limits the scalability of simulations. Therefore, we designed a two-level PM algorithm CUBE towards optimal performance in memory consumption reduction. By using the fixed-point compression technique, CUBE reduces the memory consumption per N-body particle toward 6 bytes, an order of magnitude lower than the traditional PM-based algorithms. We scaled CUBE to 512 nodes (20,480 cores) on an Intel Cascade Lake based supercomputer with $\simeq$95\% weak-scaling efficiency. This scaling test was performed in "Cosmo-$\pi$" -- a cosmological LSS simulation using $\simeq$4.4 trillion particles, tracing the evolution of the universe over $\simeq$13.7 billion years. To our best knowledge, Cosmo-$\pi$ is the largest completed cosmological N-body simulation. We believe CUBE has a huge potential to scale on exascale supercomputers for larger simulations.

arXiv comment6 pages, 5 figures. Accepted for SCALE 2020, co-located as part of the proceedings of CCGRID 2020

References in corpus (9)

• The Cosmic Linear Anisotropy Solving System (CLASS) II: Approximation schemes
• Stable clustering, the halo model and nonlinear cosmological power spectra
• High Performance P3M N-body code: CUBEP3M
• The Outer Rim Simulation: A Path to Many-Core Supercomputers
• Towards optimal parallel PM N-body codes: PMFAST
• Cosmological neutrino simulations at extreme scale
• The Q Continuum Simulation: Harnessing the Power of GPU Accelerated Supercomputers
• Differential Neutrino Condensation onto Cosmic Structure
• CUBE: An Information-optimized parallel Cosmological $N$-body Algorithm

Cited by in corpus (4)

• The Uchuu Simulations: Data Release 1 and Dark Matter Halo Concentrations
• Accurate initial conditions for cosmological N-body simulations: Minimizing truncation and discreteness errors
• Perfectly parallel cosmological simulations using spatial comoving Lagrangian acceleration
• Cosmological neutrino N-body simulations of dark matter halo