Tackling the subsampling problem to infer collective properties from limited data

Abstract

Complex systems are fascinating because their rich macroscopic properties emerge from the interaction of many simple parts. Understanding the building principles of these emergent phenomena in nature requires assessing natural complex systems experimentally. However, despite the development of large-scale data-acquisition techniques, experimental observations are often limited to a tiny fraction of the system. This spatial subsampling is particularly severe in neuroscience, where only a tiny fraction of millions or even billions of neurons can be individually recorded. Spatial subsampling may lead to significant systematic biases when inferring the collective properties of the entire system naively from a subsampled part. To overcome such biases, powerful mathematical tools have been developed in the past. In this perspective, we overview some issues arising from subsampling and review recently developed approaches to tackle the subsampling problem. These approaches enable one to assess, e.g., graph structures, collective dynamics of animals, neural network activity, or the spread of disease correctly from observing only a tiny fraction of the system. However, our current approaches are still far from having solved the subsampling problem in general, and hence we conclude by outlining what we believe are the main open challenges. Solving these challenges alongside the development of large-scale recording techniques will enable further fundamental insights into the working of complex and living systems.

20 pages, 6 figures, review article

References in corpus (12)

• Power-law distributions in empirical data
• Statistical physics of social dynamics
• Synchronization in complex networks
• The physics of higher-order interactions in complex systems
• Revealing networks from dynamics: an introduction
• Recent advances and open challenges in percolation
• Mean Field Theory For Non-Equilibrium Network Reconstruction
• Synaptic balance due to homeostatically self-organized quasicritical dynamics
• Inverse Ising problem in continuous time: A latent variable approach
• Inferring hidden states in a random kinetic Ising model: replica analysis
• Data-driven inference of hidden nodes in networks
• Variational perturbation and extended Plefka approaches to dynamics on random networks: the case of the kinetic Ising model

Cited by in corpus (1)

• Model-based assessment of sampling protocols for infectious disease genomic surveillance