Graph Database Solution for Higher Order Spatial Statistics in the Era of Big Data

Abstract

We present an algorithm for the fast computation of the general $N$-point spatial correlation functions of any discrete point set embedded within an Euclidean space of ${\mathbb{R}}^n$. Utilizing the concepts of kd-trees and graph databases, we describe how to count all possible $N$-tuples in binned configurations within a given length scale, e.g. all pairs of points or all triplets of points with side lengths $<r_{max}$. Through bench-marking we show the computational advantage of our new graph based algorithm over more traditional methods. We show that all 3-point configurations up to and beyond the Baryon Acoustic Oscillation scale ($\sim$200 Mpc in physical units) can be performed on current SDSS data in reasonable time. Finally we present the first measurements of the 4-point correlation function of $\sim$0.5 million SDSS galaxies over the redshift range $0.43<z<0.7$.

9 pages, 8 figures, submitted

References in corpus (46)

• Detection of the Baryon Acoustic Peak in the Large-Scale Correlation Function of SDSS Luminous Red Galaxies
• The Eleventh and Twelfth Data Releases of the Sloan Digital Sky Survey: Final Data from SDSS-III
• SDSS-III: Massive Spectroscopic Surveys of the Distant Universe, the Milky Way Galaxy, and Extra-Solar Planetary Systems
• The Baryon Oscillation Spectroscopic Survey of SDSS-III
• Power Spectrum Analysis of Three-Dimensional Redshift Surveys
• The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: Baryon Acoustic Oscillations in the Data Release 9 Spectroscopic Galaxy Sample
• Why your model parameter confidences might be too optimistic -- unbiased estimation of the inverse covariance matrix
• First Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Tests of Gaussianity
• Spectral Classification and Redshift Measurement for the SDSS-III Baryon Oscillation Spectroscopic Survey
• The 2dF Galaxy Redshift Survey: The bias of galaxies and the density of the Universe
• Planck 2015 results. XVII. Constraints on primordial non-Gaussianity
• Biasing and Hierarchical Statistics in Large-scale Structure
• The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: mock galaxy catalogues for the BOSS Final Data Release
• The Bispectrum as a Signature of Gravitational Instability in Redshift-Space
• The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: Analysis of potential systematics
• The Clustering of Galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: Including covariance matrix errors
• Mock galaxy catalogs using the quick particle mesh method
• The Bispectrum of IRAS Galaxies
• Detection of Baryon Acoustic Oscillation Features in the Large-Scale 3-Point Correlation Function of SDSS BOSS DR12 CMASS Galaxies
• Position-dependent power spectrum of the large-scale structure: a novel method to measure the squeezed-limit bispectrum
• Computing the Three-Point Correlation Function of Galaxies in $\mathcal{O}(N^2)$ Time
• The large-scale 3-point correlation function of the SDSS BOSS DR12 CMASS galaxies
• The Effect of Large-Scale Structure on the SDSS Galaxy Three-Point Correlation Function
• The Galaxy Power Spectrum and Bispectrum in Redshift Space
• The Three-point Correlation Function of Galaxies Determined from the Las Campanas Redshift Survey
• Fast Algorithms and Efficient Statistics: N-point Correlation Functions
• A Detection of the Baryon Acoustic Oscillation Features in the SDSS BOSS DR12 Galaxy Bispectrum
• Three-Point Correlation Functions of SDSS Galaxies: Luminosity and Color Dependence in Redshift and Projected Space
• The Projected Three-point Correlation Function: Theory and Observations
• Accelerating the 2-point and 3-point galaxy correlation functions using Fourier transforms
• The Large-Scale 3-point correlation function of SDSS Luminous Red Galaxies
• Modeling the large-scale redshift-space 3-point correlation function of galaxies
• A Practical Computational Method for the Anisotropic Redshift-Space 3-Point Correlation Function
• Clustering of Luminous Red Galaxies III: Detection of the Baryon Acoustic Peak in the 3-point Correlation Function
• Maximal compression of the redshift space galaxy power spectrum and bispectrum
• Scale Dependence of Halo Bispectrum from Non-Gaussian Initial Conditions in Cosmological N-body Simulations
• GridSPT: Grid-based calculation for perturbation theory of large-scale structure
• Enhancing BOSS bispectrum cosmological constraints with maximal compression
• Three-Point Statistics from a New Perspective
• Modelling The Redshift-Space Three-Point Correlation Function in SDSS-III
• Fast n-point correlation functions and three-point lensing application
• Experimental Cosmic Statistics II: Distribution
• Developing the 3-Point Correlation Function For the Turbulent Interstellar Medium
• Toward Accurate Modeling of the Nonlinear Matter Bispectrum: Standard Perturbation Theory and Transients from Initial Conditions
• Bispectrum as Baryon Acoustic Oscillation Interferometer
• Probing scalar tensor theories for gravity in redshift space

Cited by in corpus (23)

• Probing Parity-Violation with the Four-Point Correlation Function of BOSS Galaxies
• Measurement of Parity-Odd Modes in the Large-Scale 4-Point Correlation Function of SDSS BOSS DR12 CMASS and LOWZ Galaxies
• Testing the theory of gravity with DESI: estimators, predictions and simulation requirements
• Joint analysis of anisotropic power spectrum, bispectrum and trispectrum: application to N-body simulations
• Matter trispectrum: theoretical modelling and comparison to N-body simulations
• ENCORE: An $\mathcal{O}(N_{\mathrm{g}}^2)$ Estimator for Galaxy $N$-Point Correlation Functions
• Separate Universe Simulations with IllustrisTNG: baryonic effects on power spectrum responses and higher-order statistics
• Information Content of Higher-Order Galaxy Correlation Functions
• Efficient Computation of $N$-point Correlation Functions in $D$ Dimensions
• Integrated trispectrum detection from BOSS DR12 NGC CMASS
• GEOMAX: beyond linear compression for 3pt galaxy clustering statistics
• On the Fast Random Sampling and Other Properties of the Three Point Correlation Function in Galaxy Surveys
• Using the Mark Weighted Correlation Functions to Improve the Constraints on Cosmological Parameters
• Fast generation of mock galaxy catalogues with COLA
• A Model for the Redshift-Space Galaxy 4-Point Correlation Function
• Improving Constraint on ${\Omega }_m$ from SDSS Using Marked Correlation Functions
• Probing cosmology via the clustering of critical points
• Deep learning for cosmological parameter inference from a dark matter halo density field
• Improving SDSS cosmological constraints through $\beta$-skeleton weighted correlation functions
• Graph Theoretical Analysis of local ultraluminous infrared galaxies and quasars
• Cosmological constraints from the density gradient weighted correlation function
• Quantifying Weighted Morphological Content of Large-Scale Structures via Simulation-Based Inference
• An Optimal In-Situ Multipole Algorithm for the Isotropic Three-Point Correlation Functions