Parallel HOP: A Scalable Halo Finder for Massive Cosmological Data Sets
arXiv:1001.3411 · doi:10.1088/0067-0049/191/1/43
Abstract
Modern N-body cosmological simulations contain billions () of dark matter particles. These simulations require hundreds to thousands of gigabytes of memory, and employ hundreds to tens of thousands of processing cores on many compute nodes. In order to study the distribution of dark matter in a cosmological simulation, the dark matter halos must be identified using a halo finder, which establishes the halo membership of every particle in the simulation. The resources required for halo finding are similar to the requirements for the simulation itself. In particular, simulations have become too extensive to use commonly-employed halo finders, such that the computational requirements to identify halos must now be spread across multiple nodes and cores. Here we present a scalable-parallel halo finding method called Parallel HOP for large-scale cosmological simulation data. Based on the halo finder HOP, it utilizes MPI and domain decomposition to distribute the halo finding workload across multiple compute nodes, enabling analysis of much larger datasets than is possible with the strictly serial or previous parallel implementations of HOP. We provide a reference implementation of this method as a part of the toolkit yt, an analysis toolkit for Adaptive Mesh Refinement (AMR) data that includes complementary analysis modules. Additionally, we discuss a suite of benchmarks that demonstrate that this method scales well up to several hundred tasks and datasets in excess of particles. The Parallel HOP method and our implementation can be readily applied to any kind of N-body simulation data and is therefore widely applicable.
arXiv comment29 pages, 11 figures, 2 tables
References in corpus (21)
- The cosmological simulation code GADGET-2
- Populating a cluster of galaxies - I. Results at z=0
- Toward a halo mass function for precision cosmology: the limits of universality
- Ahf: Amiga's Halo Finder
- Precision Determination of the Mass Function of Dark Matter Halos
- Galaxies in N-body simulations: overcoming the overmerging problem
- The evolution substructure I: a new identification method
- HOP: A New Group-Finding Algorithm for N-body Simulations
- The origin and implications of dark matter anisotropic cosmic infall on ~L* haloes
- Cold Dark Matter I: The Formation of Dark Halos
- Early supersymmetric cold dark matter substructure
- VOBOZ: An Almost-Parameter-Free Halo-Finding Algorithm
- KDTREE 2: Fortran 95 and C++ software to efficiently search for near neighbors in a multi-dimensional Euclidean space
- Phase-space structures II: Hierarchical Structure Finder
- Particle-Mesh code for cosmological simulations
- A New Halo Finding Method for N-Body Simulations
- Introducing Enzo, an AMR Cosmology Application
- The Bound Mass of Substructures in Dark Matter Halos
- Fast Identification of Bound Structures in Large N-body Simulations
- A New Approach for Simulating Galaxy Cluster Properties
- Hydrodynamical Simulations of Galaxy Clusters with Galcons
Cited by in corpus (20)
- A Multi-Code Analysis Toolkit for Astrophysical Simulation Data
- Enzo: An Adaptive Mesh Refinement Code for Astrophysics
- Haloes gone MAD: The Halo-Finder Comparison Project
- Simulating cosmic structure formation with the GADGET-4 code
- The AGORA High-Resolution Galaxy Simulations Comparison Project
- Large-scale dark matter simulations
- The Nature of the Warm/Hot Intergalactic Medium I. Numerical Methods, Convergence, and OVI Absorption
- Galaxy formation in semi-analytic models and cosmological hydrodynamic zoom simulations
- Direct Numerical Simulation of Reionization II: Recombinations, Clumping Factors, and the Photon Budget for Reionization
- PHEW: a parallel segmentation algorithm for three-dimensional AMR datasets - application to structure detection in self-gravitating flows
- Evolution of primordial magnetic fields during large-scale structure formation
- The Location and Environments of Neutron Star Mergers in an Evolving Universe
- On The Road To More Realistic Galaxy Cluster Simulations: The Effects of Radiative Cooling and Thermal Feedback Prescriptions on the Observational Properties of Simulated Galaxy Clusters
- Cosmological Simulations of Isotropic Conduction in Galaxy Clusters
- Precision cosmology with baryons: non-radiative hydrodynamics of galaxy groups
- Fully Coupled Simulation of Cosmic Reionization. III. Stochastic Early Reionization by the Smallest Galaxies
- Structure in the 3D Galaxy Distribution: II. Voids and Watersheds of Local Maxima and Minima
- Hubble-Lemaître fragmentation and the path to equilibrium of merger-driven cluster formation
- Dark matter haloes: a multistream view
- Simulating Galaxy Clusters