Neural Speed Reading via Skim-RNN

Abstract

Inspired by the principles of speed reading, we introduce Skim-RNN, a recurrent neural network (RNN) that dynamically decides to update only a small fraction of the hidden state for relatively unimportant input tokens. Skim-RNN gives computational advantage over an RNN that always updates the entire hidden state. Skim-RNN uses the same input and output interfaces as a standard RNN and can be easily used instead of RNNs in existing models. In our experiments, we show that Skim-RNN can achieve significantly reduced computational cost without losing accuracy compared to standard RNNs across five different natural language tasks. In addition, we demonstrate that the trade-off between accuracy and speed of Skim-RNN can be dynamically controlled during inference time in a stable manner. Our analysis also shows that Skim-RNN running on a single CPU offers lower latency compared to standard RNNs on GPUs.

ICLR 2018

References in corpus (4)

• Recurrent Models of Visual Attention
• Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks
• Variable Computation in Recurrent Neural Networks
• Changing Model Behavior at Test-Time Using Reinforcement Learning

Cited by in corpus (12)

• How to Fine-Tune BERT for Text Classification?
• Learning Longer-term Dependencies in RNNs with Auxiliary Losses
• Dynamic Neural Networks: A Survey
• Description Based Text Classification with Reinforcement Learning
• Efficient Transformers with Dynamic Token Pooling
• Learning to Remember More with Less Memorization
• SparseIDS: Learning Packet Sampling with Reinforcement Learning
• Learning to Search in Long Documents Using Document Structure
• Long Short-Term Memory with Dynamic Skip Connections
• Interactive Machine Comprehension with Information Seeking Agents
• Image-based Natural Language Understanding Using 2D Convolutional Neural Networks
• Sequential Modelling with Applications to Music Recommendation, Fact-Checking, and Speed Reading