Discovery of Natural Language Concepts in Individual Units of CNNs
arXiv:1902.07249
Abstract
Although deep convolutional networks have achieved improved performance in many natural language tasks, they have been treated as black boxes because they are difficult to interpret. Especially, little is known about how they represent language in their intermediate layers. In an attempt to understand the representations of deep convolutional networks trained on language tasks, we show that individual units are selectively responsive to specific morphemes, words, and phrases, rather than responding to arbitrary and uninterpretable patterns. In order to quantitatively analyze such an intriguing phenomenon, we propose a concept alignment method based on how units respond to the replicated text. We conduct analyses with different architectures on multiple datasets for classification and translation tasks and provide new insights into how deep models understand natural language.
arXiv commentPublished as a conference paper at ICLR 2019
References in corpus (11)
- Sequence to Sequence Learning with Neural Networks
- Character-Aware Neural Language Models
- Visualizing and Understanding Recurrent Networks
- ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
- Modern hierarchical, agglomerative clustering algorithms
- Learning to Generate Reviews and Discovering Sentiment
- Neural Machine Translation in Linear Time
- Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
- What do Neural Machine Translation Models Learn about Morphology?
- How Transferable are Neural Networks in NLP Applications?
- Constituency Parsing with a Self-Attentive Encoder