DoGR: Disaggregated Gaussian Regression for Reproducible Analysis of Heterogeneous Data

Abstract

Quantitative analysis of large-scale data is often complicated by the presence of diverse subgroups, which reduce the accuracy of inferences they make on held-out data. To address the challenge of heterogeneous data analysis, we introduce DoGR, a method that discovers latent confounders by simultaneously partitioning the data into overlapping clusters (disaggregation) and modeling the behavior within them (regression). When applied to real-world data, our method discovers meaningful clusters and their characteristic behaviors, thus giving insight into group differences and their impact on the outcome of interest. By accounting for latent confounders, our framework facilitates exploratory analysis of noisy, heterogeneous data and can be used to learn predictive models that better generalize to new data. We provide the code to enable others to use DoGR within their data analytic workflows.

References in corpus (6)

• The Natural Selection of Bad Science
• Linear models and linear mixed effects models in R with linguistic applications
• Causal Effect Inference with Deep Latent-Variable Models
• Yule-Simpson's paradox in Galactic Archaeology
• Multiple Causal Inference with Latent Confounding
• Using Simpson's Paradox to Discover Interesting Patterns in Behavioral Data