Graph Database Solution for Higher Order Spatial Statistics in the Era of Big Data

Abstract

We present an algorithm for the fast computation of the general $N$-point spatial correlation functions of any discrete point set embedded within an Euclidean space of ${\mathbb{R}}^n$. Utilizing the concepts of kd-trees and graph databases, we describe how to count all possible $N$-tuples in binned configurations within a given length scale, e.g. all pairs of points or all triplets of points with side lengths $<r_{max}$. Through bench-marking we show the computational advantage of our new graph based algorithm over more traditional methods. We show that all 3-point configurations up to and beyond the Baryon Acoustic Oscillation scale ($\sim$200 Mpc in physical units) can be performed on current SDSS data in reasonable time. Finally we present the first measurements of the 4-point correlation function of $\sim$0.5 million SDSS galaxies over the redshift range $0.43<z<0.7$.

9 pages, 8 figures, submitted

References in corpus (7)

• Why your model parameter confidences might be too optimistic -- unbiased estimation of the inverse covariance matrix
• Position-dependent power spectrum of the large-scale structure: a novel method to measure the squeezed-limit bispectrum
• The Effect of Large-Scale Structure on the SDSS Galaxy Three-Point Correlation Function
• Three-Point Correlation Functions of SDSS Galaxies: Luminosity and Color Dependence in Redshift and Projected Space
• Maximal compression of the redshift space galaxy power spectrum and bispectrum
• Modelling The Redshift-Space Three-Point Correlation Function in SDSS-III
• Fast n-point correlation functions and three-point lensing application