A hierarchical Bayesian approach to record linkage and population size problems

Abstract

We propose and illustrate a hierarchical Bayesian approach for matching statistical records observed on different occasions. We show how this model can be profitably adopted both in record linkage problems and in capture--recapture setups, where the size of a finite population is the real object of interest. There are at least two important differences between the proposed model-based approach and the current practice in record linkage. First, the statistical model is built up on the actually observed categorical variables and no reduction (to 0--1 comparisons) of the available information takes place. Second, the hierarchical structure of the model allows a two-way propagation of the uncertainty between the parameter estimation step and the matching procedure so that no plug-in estimates are used and the correct uncertainty is accounted for both in estimating the population size and in performing the record linkage. We illustrate and motivate our proposal through a real data example and simulations.

Published in at http://dx.doi.org/10.1214/10-AOAS447 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (3)

• Describing disability through individual-level mixture models for multivariate binary data
• A hierarchical Bayesian approach to record linkage and population size problems
• Bayesian Models and Methods in Public Policy and Government Settings

Cited by in corpus (27)

• A hierarchical Bayesian approach to record linkage and population size problems
• Detecting duplicates in a homicide registry using a Bayesian partitioning approach
• Model fitting in Multiple Systems Analysis for the quantification of Modern Slavery: Classical and Bayesian approaches
• SMERED: A Bayesian Approach to Graphical Record Linkage and De-duplication
• d-blink: Distributed End-to-End Bayesian Entity Resolution
• Bayesian Propagation of Record Linkage Uncertainty into Population Size Estimation of Human Rights Violations
• Probabilistic record linkage in astronomy: Directional cross-identification and beyond
• Probit models for capture-recapture data subject to imperfect detection, individual heterogeneity and misidentification
• Scaling Bayesian Probabilistic Record Linkage with Post-Hoc Blocking: An Application to the California Great Registers
• Fast Bayesian Record Linkage for Streaming Data Contexts
• Bayesian Graphical Entity Resolution Using Exchangeable Random Partition Priors
• Unique Entity Estimation with Application to the Syrian Conflict
• Blang: Bayesian declarative modelling of general data structures and inference via algorithms based on distribution continua
• Practical Bayesian Inference for Record Linkage
• Local Log-linear Models for Capture-Recapture
• A Bayesian Multi-Layered Record Linkage Procedure to Analyze Functional Status of Medicare Patients with Traumatic Brain Injury
• Generalized Bayesian Record Linkage and Regression with Exact Error Propagation
• Bayesian Record Linkage with Variables in One File
• Fast Bayesian Record Linkage With Record-Specific Disagreement Parameters
• A Flexible Model for Record Linkage
• Analysis of Linked Files: A Missing Data Perspective
• Maximum Entropy classification for record linkage
• A Record Linkage Model Incorporating Relational Data
• Optimal Estimating Equation for Logistic Regression with Linked Data
• False Discovery estimation in Record Linkage
• On provision of UK neighbourhood population statistics beyond 2021
• Linking Sequences of Events with Sparse or No Common Occurrence across Data Sets