Multi-level Head-wise Match and Aggregation in Transformer for Textual Sequence Matching
arXiv:2001.07234
Abstract
Transformer has been successfully applied to many natural language processing tasks. However, for textual sequence matching, simple matching between the representation of a pair of sequences might bring in unnecessary noise. In this paper, we propose a new approach to sequence pair matching with Transformer, by learning head-wise matching representations on multiple levels. Experiments show that our proposed approach can achieve new state-of-the-art performance on multiple tasks that rely only on pre-computed sequence-vector-representation, such as SNLI, MNLI-match, MNLI-mismatch, QQP, and SQuAD-binary.
arXiv commentAAAI 2020, 8 pages
References in corpus (11)
- A Structured Self-attentive Sentence Embedding
- Reasoning about Entailment with Neural Attention
- A large annotated corpus for learning natural language inference
- LSTM-based Deep Learning Models for Non-factoid Answer Selection
- Learned in Translation: Contextualized Word Vectors
- Know What You Don't Know: Unanswerable Questions for SQuAD
- Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
- Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding
- A Fast Unified Model for Parsing and Sentence Understanding
- Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling
- Compare, Compress and Propagate: Enhancing Neural Architectures with Alignment Factorization for Natural Language Inference