DGA-Net Dynamic Gaussian Attention Network for Sentence Semantic Matching

Abstract

Sentence semantic matching requires an agent to determine the semantic relation between two sentences, where much recent progress has been made by the advancement of representation learning techniques and inspiration of human behaviors. Among all these methods, attention mechanism plays an essential role by selecting important parts effectively. However, current attention methods either focus on all the important parts in a static way or only select one important part at one attention step dynamically, which leaves a large space for further improvement. To this end, in this paper, we design a novel Dynamic Gaussian Attention Network (DGA-Net) to combine the advantages of current static and dynamic attention methods. More specifically, we first leverage pre-trained language model to encode the input sentences and construct semantic representations from a global perspective. Then, we develop a Dynamic Gaussian Attention (DGA) to dynamically capture the important parts and corresponding local contexts from a detailed perspective. Finally, we combine the global information and detailed local information together to decide the semantic relation of sentences comprehensively and precisely. Extensive experiments on two popular sentence semantic matching tasks demonstrate that our proposed DGA-Net is effective in improving the ability of attention mechanism.

arXiv commentAccepted by CICAI2021

References in corpus (7)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Enhanced LSTM for Natural Language Inference
• Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
• Bilateral Multi-Perspective Matching for Natural Language Sentences
• DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
• Distance-based Self-Attention Network for Natural Language Inference
• Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding