Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling

Abstract

Many natural language processing tasks solely rely on sparse dependencies between a few tokens in a sentence. Soft attention mechanisms show promising performance in modeling local/global dependencies by soft probabilities between every two tokens, but they are not effective and efficient when applied to long sentences. By contrast, hard attention mechanisms directly select a subset of tokens but are difficult and inefficient to train due to their combinatorial nature. In this paper, we integrate both soft and hard attention into one context fusion model, "reinforced self-attention (ReSA)", for the mutual benefit of each other. In ReSA, a hard attention trims a sequence for a soft self-attention to process, while the soft attention feeds reward signals back to facilitate the training of the hard one. For this purpose, we develop a novel hard attention called "reinforced sequence sampling (RSS)", selecting tokens in parallel and trained via policy gradient. Using two RSS modules, ReSA efficiently extracts the sparse dependencies between each pair of selected tokens. We finally propose an RNN/CNN-free sentence-encoding model, "reinforced self-attention network (ReSAN)", solely based on ReSA. It achieves state-of-the-art performance on both Stanford Natural Language Inference (SNLI) and Sentences Involving Compositional Knowledge (SICK) datasets.

9 pages, 2 figures; accepted at IJCAI-ECAI-18

References in corpus (7)

• ADADELTA: An Adaptive Learning Rate Method
• Neural Responding Machine for Short-Text Conversation
• DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
• Massive Exploration of Neural Machine Translation Architectures
• Learning to Compose Words into Sentences with Reinforcement Learning
• Shortcut-Stacked Sentence Encoders for Multi-Domain Inference
• Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes

Cited by in corpus (14)

• Deep Learning Based Text Classification: A Comprehensive Review
• Visual Entailment: A Novel Task for Fine-Grained Image Understanding
• Sparse Sinkhorn Attention
• Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
• ChaLearn Looking at People: IsoGD and ConGD Large-scale RGB-D Gesture Recognition
• Knowledge Enhanced Attention for Robust Natural Language Inference
• Breaking NLI Systems with Sentences that Require Simple Lexical Inferences
• Tensorized Self-Attention: Efficiently Modeling Pairwise and Global Dependencies Together
• Path-Based Contextualization of Knowledge Graphs for Textual Entailment
• Sentence Encoding with Tree-constrained Relation Networks
• Attention Boosted Sequential Inference Model
• What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks
• Three-stream network for enriched Action Recognition
• Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations