Distortion of genealogical properties when the sample is very large

Abstract

Study sample sizes in human genetics are growing rapidly, and in due course it will become routine to analyze samples with hundreds of thousands if not millions of individuals. In addition to posing computational challenges, such large sample sizes call for carefully re-examining the theoretical foundation underlying commonly-used analytical tools. Here, we study the accuracy of the coalescent, a central model for studying the ancestry of a sample of individuals. The coalescent arises as a limit of a large class of random mating models and it is an accurate approximation to the original model provided that the population size is sufficiently larger than the sample size. We develop a method for performing exact computation in the discrete-time Wright-Fisher (DTWF) model and compare several key genealogical quantities of interest with the coalescent predictions. For realistic demographic scenarios, we find that there are a significant number of multiple- and simultaneous-merger events under the DTWF model, which are absent in the coalescent by construction. Furthermore, for large sample sizes, there are noticeable differences in the expected number of rare variants between the coalescent and the DTWF model. To balance the tradeoff between accuracy and computational efficiency, we propose a hybrid algorithm that utilizes the DTWF model for the recent past and the coalescent for the more distant past. Our results demonstrate that the hybrid method with only a handful of generations of the DTWF model leads to a frequency spectrum that is quite close to the prediction of the full DTWF model.

27 pages, 2 tables, 14 figures

References in corpus (1)

• Inferring the joint demographic history of multiple populations from multidimensional SNP frequency data

Cited by in corpus (6)

• The site frequency spectrum for general coalescents
• Asymptotic behaviour of sampling and transition probabilities in coalescent models under selection and parent dependent mutations
• Weak convergence of the scaled jump chain and number of mutations of the Kingman coalescent
• Gene genealogies in haploid populations evolving according to sweepstakes reproduction
• Dirichlet approximation of equilibrium distributions in Cannings models with mutation
• High burden of private mutations due to explosive human population growth and purifying selection