Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshop
arXiv:1904.04063
Abstract
The EMNLP 2018 workshop BlackboxNLP was dedicated to resources and techniques specifically developed for analyzing and understanding the inner-workings and representations acquired by neural models of language. Approaches included: systematic manipulation of input to neural networks and investigating the impact on their performance, testing whether interpretable knowledge can be decoded from intermediate representations acquired by neural networks, proposing modifications to neural network architectures to make their knowledge state or generated output more explainable, and examining the performance of networks on simplified or formal languages. Here we review a number of representative studies in each category.
References in corpus (9)
- Categorical Reparameterization with Gumbel-Softmax
- Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
- Skip-Thought Vectors
- Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
- Learning how to explain neural networks: PatternNet and PatternAttribution
- SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
- Learning to Transduce with Unbounded Memory
- A Regularized Framework for Sparse and Structured Neural Attention
- Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis