Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference

Abstract

The RepEval 2017 Shared Task aims to evaluate natural language understanding models for sentence representation, in which a sentence is represented as a fixed-length vector with neural networks and the quality of the representation is tested with a natural language inference task. This paper describes our system (alpha) that is ranked among the top in the Shared Task, on both the in-domain test set (obtaining a 74.9% accuracy) and on the cross-domain test set (also attaining a 74.9% accuracy), demonstrating that the model generalizes well to the cross-domain data. Our model is equipped with intra-sentence gated-attention composition which helps achieve a better performance. In addition to submitting our model to the Shared Task, we have also tested it on the Stanford Natural Language Inference (SNLI) dataset. We obtain an accuracy of 85.5%, which is the best reported result on SNLI when cross-sentence attention is not allowed, the same condition enforced in RepEval 2017.

RepEval 2017 workshop paper at EMNLP 2017, Copenhagen

References in corpus (3)

• Convolutional Neural Networks for Sentence Classification
• Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
• Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering

Cited by in corpus (14)

• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• Deep Learning Based Text Classification: A Comprehensive Review
• Natural Language Inference over Interaction Space
• Distance-based Self-Attention Network for Natural Language Inference
• Dynamic Integration of Background Knowledge in Neural NLU Systems
• Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
• Leveraging Financial News for Stock Trend Prediction with Attention-Based Recurrent Neural Network
• Making Neural Machine Reading Comprehension Faster
• What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks
• Program Enhanced Fact Verification with Verbalization and Graph Attention Network
• Dropping Networks for Transfer Learning
• Learning to Embed Sentences Using Attentive Recursive Trees
• Question-Aware Sentence Gating Networks for Question and Answering
• Towards Explainable Fact Checking