CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT

Abstract

The extraction of labels from radiology text reports enables large-scale training of medical imaging models. Existing approaches to report labeling typically rely either on sophisticated feature engineering based on medical domain knowledge or manual annotations by experts. In this work, we introduce a BERT-based approach to medical image report labeling that exploits both the scale of available rule-based systems and the quality of expert annotations. We demonstrate superior performance of a biomedically pretrained BERT model first trained on annotations of a rule-based labeler and then finetuned on a small set of expert annotations augmented with automated backtranslation. We find that our final model, CheXbert, is able to outperform the previous best rules-based labeler with statistical significance, setting a new SOTA for report labeling on one of the largest datasets of chest x-rays.

arXiv commentAccepted to EMNLP 2020

References in corpus (8)

• Distributed Representations of Words and Phrases and their Compositionality
• Snorkel: Rapid Training Data Creation with Weak Supervision
• PadChest: A large chest x-ray image dataset with multi-label annotated reports
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
• Investigating Backtranslation in Neural Machine Translation
• NegBio: a high-performance tool for negation and uncertainty detection in radiology reports
• Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets
• Automated Labelling using an Attention model for Radiology reports of MRI scans (ALARM)

Cited by in corpus (9)

• Chest ImaGenome Dataset for Clinical Reasoning
• CheXpert++: Approximating the CheXpert labeler for Speed,Differentiability, and Probabilistic Output
• Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation
• Word Graph Guided Summarization for Radiology Findings
• Making Document-Level Information Extraction Right for the Right Reasons
• Structured dataset documentation: a datasheet for CheXpert
• Leveraging Deep Representations of Radiology Reports in Survival Analysis for Predicting Heart Failure Patient Mortality
• Effect of Radiology Report Labeler Quality on Deep Learning Models for Chest X-Ray Interpretation
• Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation