Annotating Social Determinants of Health Using Active Learning, and Characterizing Determinants Using Neural Event Extraction

Abstract

Social determinants of health (SDOH) affect health outcomes, and knowledge of SDOH can inform clinical decision-making. Automatically extracting SDOH information from clinical text requires data-driven information extraction models trained on annotated corpora that are heterogeneous and frequently include critical SDOH. This work presents a new corpus with SDOH annotations, a novel active learning framework, and the first extraction results on the new corpus. The Social History Annotation Corpus (SHAC) includes 4,480 social history sections with detailed annotation for 12 SDOH characterizing the status, extent, and temporal information of 18K distinct events. We introduce a novel active learning framework that selects samples for annotation using a surrogate text classification task as a proxy for a more complex event extraction task. The active learning framework successfully increases the frequency of health risk factors and improves automatic extraction of these events over undirected annotation. An event extraction model trained on SHAC achieves high extraction performance for substance use status (0.82-0.93 F1), employment status (0.81-0.86 F1), and living status type (0.81-0.93 F1) on data from three institutions.

References in corpus (3)

• Efficient Estimation of Word Representations in Vector Space
• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Multitask learning and benchmarking with clinical time series data

Cited by in corpus (7)

• Social and behavioral determinants of health in the era of artificial intelligence with electronic health records: A scoping review
• The 2022 n2c2/UW Shared Task on Extracting Social Determinants of Health
• A Study of Social and Behavioral Determinants of Health in Lung Cancer Patients Using Transformers-based Natural Language Processing Models
• SODA: A Natural Language Processing Package to Extract Social Determinants of Health for Cancer Studies
• Systematic Design and Evaluation of Social Determinants of Health Ontology (SDoHO)
• An overview of event extraction and its applications
• Backdoor Adjustment of Confounding by Provenance for Robust Text Classification of Multi-institutional Clinical Notes