Neural Attention Models for Sequence Classification: Analysis and Application to Key Term Extraction and Dialogue Act Detection

Abstract

Recurrent neural network architectures combining with attention mechanism, or neural attention model, have shown promising performance recently for the tasks including speech recognition, image caption generation, visual question answering and machine translation. In this paper, neural attention model is applied on two sequence classification tasks, dialogue act detection and key term extraction. In the sequence labeling tasks, the model input is a sequence, and the output is the label of the input sequence. The major difficulty of sequence labeling is that when the input sequence is long, it can include many noisy or irrelevant part. If the information in the whole sequence is treated equally, the noisy or irrelevant part may degrade the classification performance. The attention mechanism is helpful for sequence classification task because it is capable of highlighting important part among the entire sequence for the classification task. The experimental results show that with the attention mechanism, discernible improvements were achieved in the sequence labeling task considered here. The roles of the attention mechanism in the tasks are further analyzed and visualized in this paper.

arXiv comment5 pages, 2 figures

References in corpus (4)

• Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering
• A New Approach to Keyphrase Extraction Using Neural Networks
• The Influence of Context on Dialogue Act Recognition
• Recurrent Neural Network Encoder with Attention for Community Question Answering

Cited by in corpus (8)

• Towards Interpretable Reinforcement Learning Using Attention Augmented Agents
• Medical Time Series Classification with Hierarchical Attention-based Temporal Convolutional Networks: A Case Study of Myotonic Dystrophy Diagnosis
• Multi-level Gated Recurrent Neural Network for Dialog Act Classification
• A Dual-Attention Hierarchical Recurrent Neural Network for Dialogue Act Classification
• A Review on Dyadic Conversation Visualizations - Purposes, Data, Lens of Analysis
• Speaker-change Aware CRF for Dialogue Act Classification
• Exploring Textual and Speech information in Dialogue Act Classification with Speaker Domain Adaptation
• Hierarchical RNN with Static Sentence-Level Attention for Text-Based Speaker Change Detection