The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives
arXiv:1909.01380
Abstract
We seek to understand how the representations of individual tokens and the structure of the learned feature space evolve between layers in deep neural networks under different learning objectives. We focus on the Transformers for our analysis as they have been shown effective on various tasks, including machine translation (MT), standard left-to-right language models (LM) and masked language modeling (MLM). Previous work used black-box probing tasks to show that the representations learned by the Transformer differ significantly depending on the objective. In this work, we use canonical correlation analysis and mutual information estimators to study how information flows across Transformer layers and how this process depends on the choice of learning objective. For example, as you go from bottom to top layers, information about the past in left-to-right language models gets vanished and predictions about the future get formed. In contrast, for MLM, representations initially acquire information about the context around the token, partially forgetting the token identity and producing a more generalized token representation. The token identity then gets recreated at the top MLM layers.
arXiv commentEMNLP 2019 (camera-ready)
References in corpus (5)
- Cross-lingual Language Model Pretraining
- Opening the Black Box of Deep Neural Networks via Information
- What do Neural Machine Translation Models Learn about Morphology?
- SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
- Assessing Generative Models via Precision and Recall
Cited by in corpus (9)
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Transfusion: Understanding Transfer Learning for Medical Imaging
- Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases
- Do Vision Transformers See Like Convolutional Neural Networks?
- A Survey of Deep Learning for Scientific Discovery
- Neural Language Generation: Formulation, Methods, and Evaluation
- Differentiable Physics: A Position Piece
- Evaluating Multimodal Representations on Visual Semantic Textual Similarity
- Word Interdependence Exposes How LSTMs Compose Representations