Causality Extraction based on Self-Attentive BiLSTM-CRF with Transferred Embeddings

Abstract

Causality extraction from natural language texts is a challenging open problem in artificial intelligence. Existing methods utilize patterns, constraints, and machine learning techniques to extract causality, heavily depending on domain knowledge and requiring considerable human effort and time for feature engineering. In this paper, we formulate causality extraction as a sequence labeling problem based on a novel causality tagging scheme. On this basis, we propose a neural causality extractor with the BiLSTM-CRF model as the backbone, named SCITE (Self-attentive BiLSTM-CRF wIth Transferred Embeddings), which can directly extract cause and effect without extracting candidate causal pairs and identifying their relations separately. To address the problem of data insufficiency, we transfer contextual string embeddings, also known as Flair embeddings, which are trained on a large corpus in our task. In addition, to improve the performance of causality extraction, we introduce a multihead self-attention mechanism into SCITE to learn the dependencies between causal words. We evaluate our method on a public dataset, and experimental results demonstrate that our method achieves significant and consistent improvement compared to baselines.

arXiv comment39 pages, 11 figures, 6 tables

References in corpus (7)

• Natural Language Processing (almost) from Scratch
• On the difficulty of training Recurrent Neural Networks
• A Structured Self-attentive Sentence Embedding
• A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
• SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals
• Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Recurrent Neural Network
• Measuring the influence of mere exposure effect of TV commercial adverts on purchase behavior based on machine learning prediction models

Cited by in corpus (4)

• IDIAPers @ Causal News Corpus 2022: Efficient Causal Relation Identification Through a Prompt-based Few-shot Approach
• RTE: A Tool for Annotating Relation Triplets from Text
• Yseop at SemEval-2020 Task 5: Cascaded BERT Language Model for Counterfactual Statement Analysis
• Benchmarking LLMs for Pairwise Causal Discovery in Biomedical and Multi-Domain Contexts