Do Attention Heads in BERT Track Syntactic Dependencies?

Abstract

We investigate the extent to which individual attention heads in pretrained transformer language models, such as BERT and RoBERTa, implicitly capture syntactic dependency relations. We employ two methods---taking the maximum attention weight and computing the maximum spanning tree---to extract implicit dependency relations from the attention weights of each layer/head, and compare them to the ground-truth Universal Dependency (UD) trees. We show that, for some UD relation types, there exist heads that can recover the dependency type significantly better than baselines on parsed English text, suggesting that some self-attention heads act as a proxy for syntactic structure. We also analyze BERT fine-tuned on two datasets---the syntax-oriented CoLA and the semantics-oriented MNLI---to investigate whether fine-tuning affects the patterns of their self-attention, but we do not observe substantial differences in the overall dependency relations extracted using our methods. Our results suggest that these models have some specialist attention heads that track individual dependency types, but no generalist head that performs holistic parsing significantly better than a trivial baseline, and that analyzing attention weights directly may not reveal much of the syntactic knowledge that BERT-style models are known to learn.

References in corpus (2)

• Assessing BERT's Syntactic Abilities
• No Training Required: Exploring Random Encoders for Sentence Classification

Cited by in corpus (8)

• Grad-SAM: Explaining Transformers via Gradient Self-Attention Maps
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• Self-Explaining Structures Improve NLP Models
• Encoding Syntactic Constituency Paths for Frame-Semantic Parsing with Graph Convolutional Networks
• Does Attention Mechanism Possess the Feature of Human Reading? A Perspective of Sentiment Classification Task
• On the Importance of Local Information in Transformer Based Models
• Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation
• Learning Relation Alignment for Calibrated Cross-modal Retrieval