Corrfunc: Blazing fast correlation functions with AVX512F SIMD Intrinsics

Abstract

Correlation functions are widely used in extra-galactic astrophysics to extract insights into how galaxies occupy dark matter halos and in cosmology to place stringent constraints on cosmological parameters. A correlation function fundamentally requires computing pair-wise separations between two sets of points and then computing a histogram of the separations. Corrfunc is an existing open-source, high-performance software package for efficiently computing a multitude of correlation functions. In this paper, we will discuss the SIMD AVX512F kernels within Corrfunc, capable of processing 16 floats or 8 doubles at a time. The latest manually implemented Corrfunc AVX512F kernels show a speedup of up to $\sim 4\times$ relative to compiler-generated code for double-precision calculations. The AVX512F kernels show $\sim 1.6\times$ speedup relative to the AVX kernels and compare favorably to a theoretical maximum of $2\times$. In addition, by pruning pairs with too large of a minimum possible separation, we achieve a $\sim 5-10\%$ speedup across all the SIMD kernels. Such speedups highlight the importance of programming explicitly with SIMD vector intrinsics for complex calculations that can not be efficiently vectorized by compilers. Corrfunc is publicly available at https://github.com/manodeep/Corrfunc/.

Paper II for the Corrfunc software package, paper I is on arXiv here: arXiv:1911.03545. Appeared in the refereed proceedings for the "Second Workshop on Software Challenges to Exascale Computing"

References in corpus (2)

• Corrfunc: Blazing fast correlation functions with AVX512F SIMD Intrinsics
• Towards Accurate Modelling of Galaxy Clustering on Small Scales: Testing the Standard $\Lambda \mathrm{CDM}$ + Halo Model

Cited by in corpus (30)

• AbacusSummit: A Massive Set of High-Accuracy, High-Resolution $N$-Body Simulations
• Corrfunc --- A Suite of Blazing Fast Correlation Functions on the CPU
• The $\text{Abacus}$ Cosmological $N$-body Code
• Nearest Neighbor distributions: new statistical measures for cosmological clustering
• Dissecting and Modelling Galaxy Assembly Bias
• Cosmological cross-correlations and nearest neighbor distributions
• First Detection of the BAO Signal from Early DESI Data
• Corrfunc: Blazing fast correlation functions with AVX512F SIMD Intrinsics
• Semi-analytical covariance matrices for two-point correlation function for DESI 2024 data
• The DESI $N$-body Simulation Project -- II. Suppressing sample variance with fast simulations
• Using angular two-point correlations to self-calibrate the photometric redshift distributions of DECaLS DR9
• ELG Spectroscopic Systematics Analysis of the DESI Data Release 1
• Rosella: A mock catalogue from the P-Millennium simulation
• Galaxy Clustering in the Mira-Titan Universe I: Emulators for the redshift space galaxy correlation function and galaxy-galaxy lensing
• Detecting Baryon Acoustic Oscillations with third generation gravitational wave observatories
• Galaxy and Mass Assembly (GAMA): the clustering of galaxy groups
• Toward Accurate Modeling of Galaxy Clustering on Small Scales: Halo Model Extensions and Lingering Tension
• Constraining galaxy-halo connection with high-order statistics
• Tests of Acoustic Scale Shifts in Halo-based Mock Galaxy Catalogues
• Fast Forward Modelling of Galaxy Spatial and Statistical Distributions
• An efficient hybrid method to produce high resolution large volume dark matter simulations for semi-analytic models of reionisation
• A two-phase model of galaxy formation: III. The formation of globular clusters
• Subhalo abundance matching using progenitor mass at varying redshift: Two modes of stellar mass growth imprinted into the Subaru HSC galaxy clustering
• The power spectrum of galaxies from large to small scales: a line-intensity mapping perspective
• Convergence of halo statistics: code comparison between Rockstar and CompaSO using scale-free simulations
• Baryon Acoustic Oscillations analyses with Density-Split Statistics
• Assessing the connection between galactic conformity and assembly-type bias
• A theoretical approach to density-split clustering
• The mass and redshift dependence of halo star clustering
• Using two point correlation functions to understand the assembly histories of Milky Way-like galaxies