24.77 Pflops on a Gravitational Tree-Code to Simulate the Milky Way Galaxy with 18600 GPUs

Abstract

We have simulated, for the first time, the long term evolution of the Milky Way Galaxy using 51 billion particles on the Swiss Piz Daint supercomputer with our $N$-body gravitational tree-code Bonsai. Herein, we describe the scientific motivation and numerical algorithms. The Milky Way model was simulated for 6 billion years, during which the bar structure and spiral arms were fully formed. This improves upon previous simulations by using 1000 times more particles, and provides a wealth of new data that can be directly compared with observations. We also report the scalability on both the Swiss Piz Daint and the US ORNL Titan. On Piz Daint the parallel efficiency of Bonsai was above 95%. The highest performance was achieved with a 242 billion particle Milky Way model using 18600 GPUs on Titan, thereby reaching a sustained GPU and application performance of 33.49 Pflops and 24.77 Pflops respectively.

12 pages, 4 figures, Published in: 'Proceeding SC '14 Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis'. Gordon Bell Prize 2014 finalist

References in corpus (36)

• The Structure of Cold Dark Matter Halos
• The cosmological simulation code GADGET-2
• The Effect of the Outer Lindblad Resonance of the Galactic Bar on the Local Stellar Velocity Distribution
• Simulations of the Magellanic Stream in a First Infall Scenario
• Equilibrium Disk-Bulge-Halo Models for the Milky Way and Andromeda Galaxies
• Multi-physics simulations using a hierarchical interchangeable software interface
• Self-Perpetuating Spiral Arms in Disk Galaxies
• Dynamical Blueprints for Galaxies
• High Performance Direct Gravitational N-body Simulations on Graphics Processing Units -- II: An implementation in CUDA
• A multiphysics and multiscale software environment for modeling astrophysical systems
• A sparse octree gravitational N-body code that runs entirely on the GPU processor
• Dynamics of Non-Steady Spiral Arms in Disk Galaxies
• The dynamics of spiral arms in pure stellar disks
• Transient spirals as superposed instabilities
• Anatomy of the Bar Instability in Cuspy Dark Matter Halos
• A New Parallel N-body Gravity Solver: TPM
• GreeM : Massively Parallel TreePM Code for Large Cosmological N-body Simulations
• Understanding the spiral structure of the Milky Way using the local kinematic groups
• TreePM: A code for Cosmological N-Body Simulations
• GOTPM: A Parallel Hybrid Particle-Mesh Treecode
• High Performance Direct Gravitational N-body Simulations on Graphics Processing Unit I: An implementation in Cg
• High Performance Direct Gravitational N-body Simulations on Graphics Processing Units
• The Tree-Particle-Mesh N-body Gravity Solver
• Manifold-driven spirals in N-body barred galaxy simulations
• Kinematic groups beyond the Solar neighbourhood with RAVE
• Spiral arm pitch angle and galactic shear rate in N-body simulations of disc galaxies
• Mapping a stellar disk into a boxy bulge: The outside-in part of the Milky Way bulge formation
• Warps and Bars from the External Tidal Torques of Tumbling Dark Halos
• Relaxation in N-body simulations of disk galaxies
• 2HOT: An Improved Parallel Hashed Oct-Tree N-Body Algorithm for Cosmological Simulation
• PPPM and TreePM Methods on GRAPE Systems for Cosmological N-body Simulations
• The Effect of Many Minor Mergers on the Size Growth of Compact Quiescent Galaxies
• Simulating the universe on an intercontinental grid of supercomputers
• 4.45 Pflops Astrophysical N-Body Simulation on K computer -- The Gravitational Trillion-Body Problem
• Astrophysical Particle Simulations on Heterogeneous CPU-GPU Systems
• Visualizing Astrophysical N-body Systems

Cited by in corpus (29)

• The origin of interstellar asteroidal objects like 1I/2017 U1 'Oumuamua
• The ${\nu }^2$GC Simulations : Quantifying the Dark Side of the Universe in the Planck Cosmology
• The dynamics of stellar disks in live dark-matter halos
• Modeling the Milky Way as a Dry Galaxy
• Trimodal structure of Hercules stream explained by originating from bar resonances
• Capture of exocomets and the erosion of the Oort cloud due to stellar encounters in the Galaxy
• The rate of stellar encounters along a migrating orbit of the Sun
• GOTHIC: Gravitational oct-tree code accelerated by hierarchical time step controlling
• A GPU accelerated Barnes-Hut Tree Code for FLASH4
• A GPU-Accelerated Fast Summation Method Based on Barycentric Lagrange Interpolation and Dual Tree Traversal
• Non-intrusive hierarchical coupling strategies for multi-scale simulations in gravitational dynamics
• Radial phase spirals in the Solar neighbourhood
• Ripples spreading across the Galactic disc. Interplay of direct and indirect effects of the Sagittarius dwarf impact
• On $N$-body simulations of globular cluster stream
• Weighing the Galactic disk using phase-space spirals IV. Tests on a three-dimensional galaxy simulation
• Fast Multipole Method as a Matrix-Free Hierarchical Low-Rank Approximation
• Impact of bar resonances in the velocity-space distribution of the solar neighbourhood stars in a self-consistent $N$-body Galactic disc simulation
• Cornerstone: Octree Construction Algorithms for Scalable Particle Simulations
• Simulating a stellar contact binary merger -- I. Stellar models
• Performance analysis of parallel gravitational $N$-body codes on large GPU cluster
• DeepGalaxy: Deducing the Properties of Galaxy Mergers from Images Using Deep Neural Networks
• Delayed phase mixing in the self-gravitating Galactic disc
• The First Star-by-star $N$-body/Hydrodynamics Simulation of Our Galaxy Coupling with a Surrogate Model
• Communication Reducing Algorithms for Distributed Hierarchical N-Body Problems with Boundary Distributions
• Resolving local and global kinematic signatures of satellite mergers with billion particle simulations
• The dark matter wake of a galactic bar revealed by multichannel Singular Spectral Analysis
• Gravitational octree code performance evaluation on Volta GPU
• GPU-Enabled Particle-Particle Particle-Tree Scheme for Simulating Dense Stellar Cluster System
• Multiscale Computing in the Exascale Era