Do Attention Heads in BERT Track Syntactic Dependencies?

Abstract

We investigate the extent to which individual attention heads in pretrained transformer language models, such as BERT and RoBERTa, implicitly capture syntactic dependency relations. We employ two methods---taking the maximum attention weight and computing the maximum spanning tree---to extract implicit dependency relations from the attention weights of each layer/head, and compare them to the ground-truth Universal Dependency (UD) trees. We show that, for some UD relation types, there exist heads that can recover the dependency type significantly better than baselines on parsed English text, suggesting that some self-attention heads act as a proxy for syntactic structure. We also analyze BERT fine-tuned on two datasets---the syntax-oriented CoLA and the semantics-oriented MNLI---to investigate whether fine-tuning affects the patterns of their self-attention, but we do not observe substantial differences in the overall dependency relations extracted using our methods. Our results suggest that these models have some specialist attention heads that track individual dependency types, but no generalist head that performs holistic parsing significantly better than a trivial baseline, and that analyzing attention weights directly may not reveal much of the syntactic knowledge that BERT-style models are known to learn.

References in corpus (4)

• Assessing BERT's Syntactic Abilities
• Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks
• No Training Required: Exploring Random Encoders for Sentence Classification
• Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis

Cited by in corpus (17)

• Molecule Attention Transformer
• Grad-SAM: Explaining Transformers via Gradient Self-Attention Maps
• Artificial Text Detection via Examining the Topology of Attention Maps
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• Self-Explaining Structures Improve NLP Models
• Acceptability Judgements via Examining the Topology of Attention Maps
• Linguistic Dependencies and Statistical Dependence
• Encoding Syntactic Constituency Paths for Frame-Semantic Parsing with Graph Convolutional Networks
• Attention as a Guide for Simultaneous Speech Translation
• Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
• Shaking Syntactic Trees on the Sesame Street: Multilingual Probing with Controllable Perturbations
• On the Importance of Local Information in Transformer Based Models
• Does Attention Mechanism Possess the Feature of Human Reading? A Perspective of Sentiment Classification Task
• Learning Relation Alignment for Calibrated Cross-modal Retrieval
• Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation
• Have Attention Heads in BERT Learned Constituency Grammar?
• On Commonsense Cues in BERT for Solving Commonsense Tasks