HACC: Simulating Sky Surveys on State-of-the-Art Supercomputing Architectures
arXiv:1410.2805 · doi:10.1016/j.newast.2015.06.003
Abstract
Current and future surveys of large-scale cosmic structure are associated with a massive and complex datastream to study, characterize, and ultimately understand the physics behind the two major components of the 'Dark Universe', dark energy and dark matter. In addition, the surveys also probe primordial perturbations and carry out fundamental measurements, such as determining the sum of neutrino masses. Large-scale simulations of structure formation in the Universe play a critical role in the interpretation of the data and extraction of the physics of interest. Just as survey instruments continue to grow in size and complexity, so do the supercomputers that enable these simulations. Here we report on HACC (Hardware/Hybrid Accelerated Cosmology Code), a recently developed and evolving cosmology N-body code framework, designed to run efficiently on diverse computing architectures and to scale to millions of cores and beyond. HACC can run on all current supercomputer architectures and supports a variety of programming models and algorithms. It has been demonstrated at scale on Cell- and GPU-accelerated systems, standard multi-core node clusters, and Blue Gene systems. HACC's design allows for ease of portability, and at the same time, high levels of sustained performance on the fastest supercomputers available. We present a description of the design philosophy of HACC, the underlying algorithms and code structure, and outline implementation details for several specific architectures. We show selected accuracy and performance results from some of the largest high resolution cosmological simulations so far performed, including benchmarks evolving more than 3.6 trillion particles.
23 pages, 20 figures
References in corpus (6)
- Wide-Field InfrarRed Survey Telescope-Astrophysics Focused Telescope Assets WFIRST-AFTA 2015 Report
- A primer on hierarchical galaxy formation: the semi-analytical approach
- J-PAS: The Javalambre-Physics of the Accelerated Universe Astrophysical Survey
- Cosmic Calibration: Constraints from the Matter Power Spectrum and the Cosmic Microwave Background
- Simulation techniques for cosmological simulations
- Cosmic Calibration
Cited by in corpus (109)
- Simulating cosmic structure formation with the GADGET-4 code
- The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: BAO and RSD measurements from the anisotropic power spectrum of the Quasar sample between redshift 0.8 and 2.2
- An accurate physical model for halo concentrations
- AbacusSummit: A Massive Set of High-Accuracy, High-Resolution -Body Simulations
- The SPTpol Extended Cluster Survey
- The BlueTides Simulation: First Galaxies and Reionization
- Euclid preparation: II. The EuclidEmulator -- A tool to compute the cosmology dependence of the nonlinear matter power spectrum
- Large-scale dark matter simulations
- The clustering of the SDSS-IV extended Baryon Oscillation Spectroscopic Survey DR14 quasar sample: measurement of the growth rate of structure from the anisotropic correlation function between redshift 0.8 and 2.2
- The clustering of the SDSS-IV extended Baryon Oscillation Spectroscopic Survey DR14 quasar sample: structure growth rate measurement from the anisotropic quasar power spectrum in the redshift range
- The Mira-Titan Universe II: Matter Power Spectrum Emulation
- The Mira-Titan Universe: Precision Predictions for Dark Energy Surveys
- The ASTRID Simulation: Galaxy Formation and Reionization
- Halo Profiles and the Concentration-Mass Relation for a ΛCDM Universe
- Detection of the kinematic Sunyaev-Zel'dovich effect with DES Year 1 and SPT
- CosmoDC2: A Synthetic Sky Catalog for Dark Energy Science with LSST
- SWIFT: A modern highly-parallel gravity and smoothed particle hydrodynamics solver for astrophysical and cosmological applications
- The Outer Rim Simulation: A Path to Many-Core Supercomputers
- Revealing the galaxy-halo connection in IllustrisTNG
- CRKSPH - A Conservative Reproducing Kernel Smoothed Particle Hydrodynamics Scheme
- UNIT project: Universe -body simulations for the Investigation of Theoretical models from galaxy surveys
- Dark Energy Survey Year 1 Results: Calibration of Cluster Mis-centering in the redMaPPer Catalogs
- The LSST DESC DC2 Simulated Sky Survey
- Simulations and symmetries
- The clustering of the SDSS-IV extended Baryon Oscillation Spectroscopic Survey DR14 quasar sample: anisotropic clustering analysis in configuration-space
- Dark matter statistics for large galaxy catalogs: power spectra and covariance matrices
- Cosmological neutrino simulations at extreme scale
- The Completed SDSS-IV Extended Baryon Oscillation Spectroscopic Survey: N-body Mock Challenge for the Quasar Sample
- Mocking the Weak Lensing universe: the LensTools python computing package
- The completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Cosmological implications from multi-tracer BAO analysis with galaxies and voids
- A Practical Computational Method for the Anisotropic Redshift-Space 3-Point Correlation Function
- Accurate mass and velocity functions of dark matter halos
- CARPool: fast, accurate computation of large-scale structure statistics by pairing costly and cheap cosmological simulations
- The Mira-Titan Universe IV. High Precision Power Spectrum Emulation
- The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: N-body Mock Challenge for the eBOSS Emission Line Galaxy Sample
- Understanding GPU-Based Lossy Compression for Extreme-Scale Cosmological Simulations
- Euclid. V. The Flagship galaxy mock catalogue: a comprehensive simulation for the Euclid mission
- The Completed SDSS-IV Extended Baryon Oscillation Spectroscopic Survey: N-body Mock Challenge for Galaxy Clustering Measurements
- Aemulus : Precise Predictions for Matter and Biased Tracer Power Spectra in the Presence of Neutrinos
- Simulations of the Pairwise Kinematic Sunyaev-Zel'dovich Signal
- CosmoGridV1: a simulated CDM theory prediction for map-level cosmological inference
- On the road to percent accuracy III: non-linear reaction of the matter power spectrum to massive neutrinos
- PICS: Simulations of Strong Gravitational Lensing in Galaxy Clusters
- Redshift-space distortions in massive neutrino and evolving dark energy cosmologies
- The clustering of the SDSS-IV extended Baryon Oscillation Spectroscopic Survey DR14 quasar sample: measuring the evolution of the growth rate using redshift space distortions between redshift 0.8 and 2.2
- Enabling particle applications for exascale computing platforms
- Cosmology with photometric weak lensing surveys: constraints with redshift tomography of convergence peaks and moments
- Highly parallelisable simulations of time-dependent viscoplastic fluid flow simulations with structured adaptive mesh refinement
- Precision Redshift-Space Galaxy Power Spectra using Zel'dovich Control Variates
- Euclid: Modelling massive neutrinos in cosmology -- a code comparison
- HACC Cosmological Simulations: First Data Release
- Machine Learning methods to estimate observational properties of galaxy clusters in large volume cosmological N-body simulations
- Generating Synthetic Cosmological Data with GalSampler
- Effects of long-wavelength fluctuations in large galaxy surveys
- Modelling galaxy clustering in redshift space with a Lagrangian bias formalism and -body simulations
- The Last Journey. I. An Extreme-Scale Simulation on the Mira Supercomputer
- The impact of line-of-sight structures on measuring with strong lensing time-delays
- Differentiable Cosmological Simulation with Adjoint Method
- Adaptive Configuration of In Situ Lossy Compression for Cosmology Simulations via Fine-Grained Rate-Quality Modeling
- An embedded boundary approach for efficient simulations of viscoplastic fluids in three dimensions
- Modeling the Lyman- forest with Eulerian and SPH hydrodynamical methods
- A Hybrid Fast Multipole Method for Cosmological N-body Simulations
- Improving MPI Collective I/O Performance With Intra-node Request Aggregation
- The Borg Cube Simulation: Cosmological Hydrodynamics with CRK-SPH
- Combined full shape analysis of BOSS galaxies and eBOSS quasars using an iterative emulator
- Reducing the Variance of Redshift Space Distortion Measurements from Mock Galaxy Catalogues with Different Lines of Sight
- The importance of secondary halos for strong lensing in massive galaxy clusters across redshift
- CMASS galaxy sample and the ontological status of the cosmological principle
- Farpoint: A High-Resolution Cosmology Simulation at the Gigaparsec Scale
- Simulating Hydrodynamics in Cosmology with CRK-HACC
- Efficient Construction of Mock Catalogs for Baryon Acoustic Oscillation Surveys
- Bayesian Control Variates for optimal covariance estimation with pairs of simulations and surrogates
- Galaxy Clustering in the Mira-Titan Universe I: Emulators for the redshift space galaxy correlation function and galaxy-galaxy lensing
- Wafer-Scale Fast Fourier Transforms
- The Ultramarine Simulation: properties of dark matter haloes before redshift 5.5
- Efficient Mass Estimate at the Core of Strong Lensing Galaxy Clusters Using the Einstein Radius
- Why are we still using 3D masses for cluster cosmology?
- Pairwise kSZ signal extraction efficacy and optical depth estimation
- Fast tree-based algorithms for DBSCAN for low-dimensional data on GPUs
- Improving initialization and evolution accuracy of cosmological neutrino simulations
- The Last Journey. II. SMACC -- Subhalo Mass-loss Analysis using Core Catalogs
- BullFrog: Multi-step perturbation theory as a time integrator for cosmological simulations
- A single-tree algorithm to compute the Euclidean minimum spanning tree on GPUs
- Exploring the Role of Machine Learning in Scientific Workflows: Opportunities and Challenges
- Core Mass Estimates in Strong Lensing Galaxy Clusters Using a Single-Halo Lens Model
- PhotoNs-GPU:A GPU accelerated cosmological simulation code
- Numerical Discreteness Errors in Multi-Species Cosmological N-body Simulations
- A Machine Learning Approach to Correct for Mass Resolution Effects in Simulated Halo Clustering Statistics
- Differentiable Cosmological Hydrodynamics for Field-Level Inference and High Dimensional Parameter Constraints
- Covariance matrices for variance-suppressed simulations
- Perturbation-theory informed integrators for cosmological simulations
- Advances in ArborX to support exascale applications
- ASCR/HEP Exascale Requirements Review Report
- Evolution of the Deterministic Collapse Barrier of the Field Clusters as a Probe of Cosmology
- PANDORA: A Parallel Dendrogram Construction Algorithm for Single Linkage Clustering on GPU
- kSZ Pairwise Velocity Reconstruction with Machine Learning
- Building Halo Merger Trees from the Q Continuum Simulation
- Exploration of Pattern-Matching Techniques for Lossy Compression on Cosmology Simulation Data Sets
- Lightcone Construction for HACC Cosmological Simulations with LANTERN
- The Completed SDSS-IV Extended Baryon Oscillation Spectroscopic Survey: Growth rate of structure measurement from cosmic voids
- Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP
- A Conditional Abundance Matching Method of Extending Simulated Halo Merger Trees to Resolve Low-Mass Progenitors and Sub-halos
- Evolution of Cluster Alignments as Evidence of Large-scale Structure Formation in the Universe
- On the possibility of Baryon Acoustic Oscillation measurements at redshift with the Roman Space Telescope
- Exploring Trade-offs in Dynamic Task Triggering for Loosely Coupled Scientific Workflows
- Efficient Computation of Large-Scale Statistical Solutions to Incompressible Fluid Flows
- In-Depth Exploration of Single-Snapshot Lossy Compression Techniques for N-Body Simulations
- Fast Integrity Verification for High-Speed File Transfers
- Improved Methods for Estimating Peculiar Velocity Correlation Functions Using Volume Weighting