Persistence pays off: Paying Attention to What the LSTM Gating Mechanism Persists

Giancarlo D. Salton, John D. Kelleher

Language Models (LMs) are important components in several Natural Language Processing systems. Recurrent Neural Network LMs composed of LSTM units, especially those augmented with…