Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)

Abstract

Neural networks models for NLP are typically implemented without the explicit encoding of language rules and yet they are able to break one performance record after another. This has generated a lot of research interest in interpreting the representations learned by these networks. We propose here a novel interpretation approach that relies on the only processing system we have that does understand language: the human brain. We use brain imaging recordings of subjects reading complex natural text to interpret word and sequence embeddings from 4 recent NLP models - ELMo, USE, BERT and Transformer-XL. We study how their representations differ across layer depth, context length, and attention type. Our results reveal differences in the context-related representations across these models. Further, in the transformer models, we find an interaction between layer depth and context length, and between layer depth and attention type. We finally hypothesize that altering BERT to better align with brain recordings would enable it to also better understand language. Probing the altered BERT using syntactic NLP tasks reveals that the model with increased brain-alignment outperforms the original model. Cognitive neuroscientists have already begun using NLP networks to study the brain, and this work closes the loop to allow the interaction between NLP and cognitive neuroscience to be a true cross-pollination.

arXiv commentNeurIPS 2019

References in corpus (7)

• Universal Sentence Encoder
• Skip-Thought Vectors
• Assessing BERT's Syntactic Abilities
• Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
• Regularized brain reading with shrinkage and smoothing
• Targeted Syntactic Evaluation of Language Models
• Recurrent Neural Networks as Weighted Language Recognizers

Cited by in corpus (14)

• Interpreting Deep Learning-Based Networking Systems
• Decoding EEG Brain Activity for Multi-Modal Natural Language Processing
• Disentangling Syntax and Semantics in the Brain with Deep Networks
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• Neural Language Taskonomy: Which NLP Tasks are the most Predictive of fMRI Brain Activity?
• The Temporal Structure of Language Processing in the Human Brain Corresponds to The Layered Hierarchy of Deep Language Models
• Beyond linear regression: mapping models in cognitive neuroscience should align with research goals
• Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)
• Interpretable Modeling of Deep Reinforcement Learning Driven Scheduling
• Structural Similarities Between Language Models and Neural Response Measurements
• Same Cause; Different Effects in the Brain
• Data-driven models and computational tools for neurolinguistics: a language technology perspective
• Divergences between Language Models and Human Brains
• Long-range and hierarchical language predictions in brains and algorithms