A Multi-Size Neural Network with Attention Mechanism for Answer Selection
arXiv:2105.03278
Abstract
Semantic matching is of central significance to the answer selection task which aims to select correct answers for a given question from a candidate answer pool. A useful method is to employ neural networks with attention to generate sentences representations in a way that information from pair sentences can mutually influence the computation of representations. In this work, an effective architecture,multi-size neural network with attention mechanism (AM-MSNN),is introduced into the answer selection task. This architecture captures more levels of language granularities in parallel, because of the various sizes of filters comparing with single-layer CNN and multi-layer CNNs. Meanwhile it extends the sentence representations by attention mechanism, thus containing more information for different types of questions. The empirical study on three various benchmark tasks of answer selection demonstrates the efficacy of the proposed model in all the benchmarks and its superiority over competitors. The experimental results show that (1) multi-size neural network (MSNN) is a more useful method to capture abstract features on different levels of granularities than single/multi-layer CNNs; (2) the attention mechanism (AM) is a better strategy to derive more informative representations; (3) AM-MSNN is a better architecture for the answer selection task for the moment.
References in corpus (6)
- A Structured Self-attentive Sentence Embedding
- Convolutional Neural Network Architectures for Matching Natural Language Sentences
- LSTM-based Deep Learning Models for Non-factoid Answer Selection
- Deep Learning for Answer Sentence Selection
- Attentive Pooling Networks
- Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems