Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
arXiv:2003.11515
Abstract
In this work, we examine the extent to which embeddings may encode marginalized populations differently, and how this may lead to a perpetuation of biases and worsened performance on clinical tasks. We pretrain deep embedding models (BERT) on medical notes from the MIMIC-III hospital dataset, and quantify potential disparities using two approaches. First, we identify dangerous latent relationships that are captured by the contextual word embeddings using a fill-in-the-blank method with text from real clinical notes and a log probability bias score quantification. Second, we evaluate performance gaps across different definitions of fairness on over 50 downstream clinical prediction tasks that include detection of acute and chronic conditions. We find that classifiers trained from BERT representations exhibit statistically significant differences in performance, often favoring the majority group with regards to gender, language, ethnicity, and insurance status. Finally, we explore shortcomings of using adversarial debiasing to obfuscate subgroup information in contextual word embeddings, and recommend best practices for such deep embedding models in clinical settings.
arXiv commentAccepted at ACM CHIL 2020 (Spotlight)
References in corpus (20)
- Distributed Representations of Words and Phrases and their Compositionality
- BioBERT: a pre-trained biomedical language representation model for biomedical text mining
- Equality of Opportunity in Supervised Learning
- Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
- Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
- Publicly Available Clinical BERT Embeddings
- ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission
- Enhancing Clinical Concept Extraction with Contextual Embeddings
- Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
- A Review of Challenges and Opportunities in Machine Learning for Health
- Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them
- Fairness Without Demographics in Repeated Loss Minimization
- Learning Adversarially Fair and Transferable Representations
- Censoring Representations with an Adversary
- SciBERT: A Pretrained Language Model for Scientific Text
- Gender Bias in Contextualized Word Embeddings
- Learning Gender-Neutral Word Embeddings
- Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations
- Evaluating the Underlying Gender Bias in Contextualized Word Embeddings
- Adversarial Removal of Demographic Attributes from Text Data