ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs

Abstract

How to model a pair of sentences is a critical issue in many NLP tasks such as answer selection (AS), paraphrase identification (PI) and textual entailment (TE). Most prior work (i) deals with one individual task by fine-tuning a specific system; (ii) models each sentence's representation separately, rarely considering the impact of the other sentence; or (iii) relies fully on manually designed, task-specific linguistic features. This work presents a general Attention Based Convolutional Neural Network (ABCNN) for modeling a pair of sentences. We make three contributions. (i) ABCNN can be applied to a wide variety of tasks that require modeling of sentence pairs. (ii) We propose three attention schemes that integrate mutual influence between sentences into CNN; thus, the representation of each sentence takes into consideration its counterpart. These interdependent sentence pair representations are more powerful than isolated sentence representations. (iii) ABCNN achieves state-of-the-art performance on AS, PI and TE tasks.

TACL Camera-ready

References in corpus (12)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Recurrent Models of Visual Attention
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• DRAW: A Recurrent Neural Network For Image Generation
• Reasoning about Entailment with Neural Attention
• A Convolutional Neural Network for Modelling Sentences
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• LSTM-based Deep Learning Models for Non-factoid Answer Selection
• Deep Learning for Answer Sentence Selection
• ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
• Applying Deep Learning to Answer Selection: A Study and An Open Task
• Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network

Cited by in corpus (21)

• Bilateral Multi-Perspective Matching for Natural Language Sentences
• Neural Models for Information Retrieval
• Adversarial Neural Machine Translation
• High-Order Attention Models for Visual Question Answering
• Learning Loss Functions for Semi-supervised Learning via Discriminative Adversarial Networks
• Modelling Sentence Pairs with Tree-structured Attentive Encoder
• Learning Visual Knowledge Memory Networks for Visual Question Answering
• Adaptability of Neural Networks on Varying Granularity IR Tasks
• Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax
• Modelling Interaction of Sentence Pair with coupled-LSTMs
• Can Neural Networks Understand Logical Entailment?
• A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots
• M${}^2$S-Net: Multi-Modal Similarity Metric Learning based Deep Convolutional Network for Answer Selection
• Multi-Perspective Relevance Matching with Hierarchical ConvNets for Social Media Search
• Effective Parallel Corpus Mining using Bilingual Sentence Embeddings
• A Sensitivity Analysis of Attention-Gated Convolutional Neural Networks for Sentence Classification
• A Novel Deep Learning Method for Textual Sentiment Analysis
• Dropping Networks for Transfer Learning
• Hierarchical Gated Recurrent Neural Tensor Network for Answer Triggering
• Discriminative Information Retrieval for Knowledge Discovery
• Learning to Focus when Ranking Answers