Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering

Abstract

In this paper, we analyze several neural network designs (and their variations) for sentence pair modeling and compare their performance extensively across eight datasets, including paraphrase identification, semantic textual similarity, natural language inference, and question answering tasks. Although most of these models have claimed state-of-the-art performance, the original papers often reported on only one or two selected datasets. We provide a systematic study and show that (i) encoding contextual information by LSTM and inter-sentence interactions are critical, (ii) Tree-LSTM does not help as much as previously claimed but surprisingly improves performance on Twitter datasets, (iii) the Enhanced Sequential Inference Model is the best so far for larger datasets, while the Pairwise Word Interaction Model achieves the best performance when less data is available. We release our implementations as an open-source toolkit.

arXiv comment13 pages; accepted to COLING 2018

References in corpus (8)

• Enhanced LSTM for Natural Language Inference
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• A large annotated corpus for learning natural language inference
• Natural Language Inference over Interaction Space
• Bilateral Multi-Perspective Matching for Natural Language Sentences
• DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
• Shortcut-Stacked Sentence Encoders for Multi-Domain Inference
• A Continuously Growing Dataset of Sentential Paraphrases

Cited by in corpus (15)

• DocBERT: BERT for Document Classification
• Pretrained Transformers for Text Ranking: BERT and Beyond
• CoaCor: Code Annotation for Code Retrieval with Reinforcement Learning
• Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks
• Using Prior Knowledge to Guide BERT's Attention in Semantic Textual Matching Tasks
• Identifying Machine-Paraphrased Plagiarism
• A Multi-cascaded Model with Data Augmentation for Enhanced Paraphrase Detection in Short Texts
• Hierarchical Character Embeddings: Learning Phonological and Semantic Representations in Languages of Logographic Origin using Recursive Neural Networks
• Dialogue Natural Language Inference
• Inter-BIN: Interaction-based Cross-architecture IoT Binary Similarity Comparison
• LadRa-Net: Locally-Aware Dynamic Re-read Attention Net for Sentence Semantic Matching
• Attention Boosted Sequential Inference Model
• Generalizing Natural Language Analysis through Span-relation Representations
• Polly Want a Cracker: Analyzing Performance of Parroting on Paraphrase Generation Datasets
• PARADE: A New Dataset for Paraphrase Identification Requiring Computer Science Domain Knowledge