Non-local Recurrent Neural Memory for Supervised Sequence Modeling
arXiv:1908.09535
Abstract
Typical methods for supervised sequence modeling are built upon the recurrent neural networks to capture temporal dependencies. One potential limitation of these methods is that they only model explicitly information interactions between adjacent time steps in a sequence, hence the high-order interactions between nonadjacent time steps are not fully exploited. It greatly limits the capability of modeling the long-range temporal dependencies since one-order interactions cannot be maintained for a long term due to information dilution and gradient vanishing. To tackle this limitation, we propose the Non-local Recurrent Neural Memory (NRNM) for supervised sequence modeling, which performs non-local operations to learn full-order interactions within a sliding temporal block and models global interactions between blocks in a gated recurrent manner. Consequently, our model is able to capture the long-range dependencies. Besides, the latent high-level features contained in high-order interactions can be distilled by our model. We demonstrate the merits of our NRNM on two different tasks: action recognition and sentiment analysis.
arXiv commentAccepted by ICCV 2019, Oral
References in corpus (9)
- The Kinetics Human Action Video Dataset
- Dynamic Memory Networks for Visual and Textual Question Answering
- Semi-supervised Sequence Learning
- Learning to Generate Reviews and Discovering Sentiment
- Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
- TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency
- Improving Neural Language Models with a Continuous Cache
- Higher Order Recurrent Neural Networks
- GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations