How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representations

Abstract

Bidirectional Encoder Representations from Transformers (BERT) reach state-of-the-art results in a variety of Natural Language Processing tasks. However, understanding of their internal functioning is still insufficient and unsatisfactory. In order to better understand BERT and other Transformer-based models, we present a layer-wise analysis of BERT's hidden states. Unlike previous research, which mainly focuses on explaining Transformer models by their attention weights, we argue that hidden states contain equally valuable information. Specifically, our analysis focuses on models fine-tuned on the task of Question Answering (QA) as an example of a complex downstream task. We inspect how QA models transform token vectors in order to find the correct answer. To this end, we apply a set of general and QA-specific probing tasks that reveal the information stored in each representation layer. Our qualitative analysis of hidden state visualizations provides additional insights into BERT's reasoning process. Our results show that the transformations within BERT go through phases that are related to traditional pipeline tasks. The system can therefore implicitly incorporate task-specific information into its token representations. Furthermore, our analysis reveals that fine-tuning has little impact on the models' semantic abilities and that prediction errors can be recognized in the vector representations of even early layers.

arXiv commentAccepted at CIKM 2019

Cited by in corpus (8)

• Exploring the State of the Art in Legal QA Systems
• Unifying Two-Stream Encoders with Transformers for Cross-Modal Retrieval
• Federated Few-Shot Learning for Mobile NLP
• Towards Practical Few-shot Federated NLP
• The Impact of Cross-Lingual Adjustment of Contextual Word Representations on Zero-Shot Transfer
• Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-Encoders
• A Pairwise Probe for Understanding BERT Fine-Tuning on Machine Reading Comprehension
• Human-Computer Interaction and Visualization in Natural Language Generation Models: Applications, Challenges, and Opportunities