Understanding Recurrent Neural State Using Memory Signatures

Abstract

We demonstrate a network visualization technique to analyze the recurrent state inside the LSTMs/GRUs used commonly in language and acoustic models. Interpreting intermediate state and network activations inside end-to-end models remains an open challenge. Our method allows users to understand exactly how much and what history is encoded inside recurrent state in grapheme sequence models. Our procedure trains multiple decoders that predict prior input history. Compiling results from these decoders, a user can obtain a signature of the recurrent kernel that characterizes its memory behavior. We demonstrate this method's usefulness in revealing information divergence in the bases of recurrent factorized kernels, visualizing the character-level differences between the memory of n-gram and recurrent language models, and extracting knowledge of history encoded in the layers of grapheme-based end-to-end ASR networks.

arXiv commentAccepted to 2018 IEEE International Conference on Acoustics, Speech and Signal Processing

References in corpus (8)

• Methods for Interpreting and Understanding Deep Neural Networks
• Visualizing and Understanding Recurrent Networks
• Learning how to explain neural networks: PatternNet and PatternAttribution
• Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
• Visualizing and Understanding Neural Models in NLP
• Automatic Rule Extraction from Long Short Term Memory Networks
• Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models
• N-gram Language Modeling using Recurrent Neural Network Estimation