exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models

Abstract

Large language models can produce powerful contextual representations that lead to improvements across many NLP tasks. Since these models are typically guided by a sequence of learned self attention mechanisms and may comprise undesired inductive biases, it is paramount to be able to explore what the attention has learned. While static analyses of these models lead to targeted insights, interactive tools are more dynamic and can help humans better gain an intuition for the model-internal reasoning process. We present exBERT, an interactive tool named after the popular BERT language model, that provides insights into the meaning of the contextual representations by matching a human-specified input to similar contexts in a large annotated dataset. By aggregating the annotations of the matching similar contexts, exBERT helps intuitively explain what each attention-head has learned.

References in corpus (2)

• CTRL: A Conditional Transformer Language Model for Controllable Generation
• What do you learn from context? Probing for sentence structure in contextualized word representations

Cited by in corpus (11)

• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• Pre-trained Models for Natural Language Processing: A Survey
• LXMERT: Learning Cross-Modality Encoder Representations from Transformers
• On the Explainability of Natural Language Processing Deep Models
• BERT Learns (and Teaches) Chemistry
• DeepLens: Interactive Out-of-distribution Data Detection in NLP Models
• What Would You Ask the Machine Learning Model? Identification of User Needs for Model Explanations Based on Human-Model Conversations
• Towards Grad-CAM Based Explainability in a Legal Text Processing Pipeline
• Semantic Representation and Inference for NLP
• Multi-Head Self-Attention with Role-Guided Masks
• Understood in Translation, Transformers for Domain Understanding