Explicit Contextual Semantics for Text Comprehension

Abstract

Who did what to whom is a major focus in natural language understanding, which is right the aim of semantic role labeling (SRL) task. Despite of sharing a lot of processing characteristics and even task purpose, it is surprisingly that jointly considering these two related tasks was never formally reported in previous work. Thus this paper makes the first attempt to let SRL enhance text comprehension and inference through specifying verbal predicates and their corresponding semantic roles. In terms of deep learning models, our embeddings are enhanced by explicit contextual semantic role labels for more fine-grained semantics. We show that the salient labels can be conveniently added to existing models and significantly improve deep learning models in challenging text comprehension tasks. Extensive experiments on benchmark machine reading comprehension and inference datasets verify that the proposed semantic learning helps our system reach new state-of-the-art over strong baselines which have been enhanced by well pretrained language models from the latest progress.

Proceedings of the 33nd Pacific Asia Conference on Language, Information and Computation (PACLIC 33)

References in corpus (13)

• Teaching Machines to Read and Comprehend
• Training Very Deep Networks
• Enhanced LSTM for Natural Language Inference
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
• The Natural Language Decathlon: Multitask Learning as Question Answering
• Know What You Don't Know: Unanswerable Questions for SQuAD
• Modeling Multi-turn Conversation with Deep Utterance Aggregation
• Deep Enhanced Representation for Implicit Discourse Relation Recognition
• One-shot Learning for Question-Answering in Gaokao History Challenge
• Subword-augmented Embedding for Cloze Reading Comprehension
• Head-Driven Phrase Structure Grammar Parsing on Penn Treebank
• Lattice-Based Transformer Encoder for Neural Machine Translation

Cited by in corpus (11)

• Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
• Self-Explaining Structures Improve NLP Models
• Semantics-aware BERT for Language Understanding
• Dependency or Span, End-to-End Uniform Semantic Role Labeling
• LIMIT-BERT : Linguistic Informed Multi-Task BERT
• Path-Based Contextualization of Knowledge Graphs for Textual Entailment
• Effective Subword Segmentation for Text Comprehension
• High-order Semantic Role Labeling
• Semi-supervised Visual Feature Integration for Pre-trained Language Models
• ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference
• BERT-DRE: BERT with Deep Recursive Encoder for Natural Language Sentence Matching