Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

Abstract

A fundamental goal of scientific research is to learn about causal relationships. However, despite its critical role in the life and social sciences, causality has not had the same importance in Natural Language Processing (NLP), which has traditionally placed more emphasis on predictive tasks. This distinction is beginning to fade, with an emerging area of interdisciplinary research at the convergence of causal inference and language processing. Still, research on causality in NLP remains scattered across domains without unified definitions, benchmark datasets and clear articulations of the challenges and opportunities in the application of causal inference to the textual domain, with its unique properties. In this survey, we consolidate research across academic areas and situate it in the broader NLP landscape. We introduce the statistical challenge of estimating causal effects with text, encompassing settings where text is used as an outcome, treatment, or to address confounding. In addition, we explore potential uses of causal inference to improve the robustness, fairness, and interpretability of NLP models. We thus provide a unified overview of causal inference for the NLP community.

Accepted to Transactions of the Association for Computational Linguistics (TACL)

References in corpus (15)

• Shortcut Learning in Deep Neural Networks
• Equality of Opportunity in Supervised Learning
• Domain Generalization via Invariant Feature Representation
• Attention is not Explanation
• Towards a Critical Race Methodology in Algorithmic Fairness
• On Causal and Anticausal Learning
• Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
• In Search of Lost Domain Generalization
• Quantifying the Causal Effects of Conversational Tendencies
• Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models
• On Calibration and Out-of-domain Generalization
• Fairness and Robustness in Invariant Learning: A Case Study in Toxicity Classification
• Explaining The Efficacy of Counterfactually Augmented Data
• Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests
• TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling

Cited by in corpus (2)

• Local Interpretations for Explainable Natural Language Processing: A Survey
• Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective