125 citations · 584 across the 45 of their papers we have counts for
9 papers · 1 filter
BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance
R. Thomas McCoy, Junghyun Min, Tal Linzen
If the same neural network architecture is trained multiple times on the same dataset, will it make similar linguistic generalizations across runs? To study this question, we fine-…
Discovering the Compositional Structure of Vector Representations with Role Learning Networks
Paul Soulos, Tom McCoy, Tal Linzen +1
How can neural networks perform so well on compositional tasks even though they lack explicit compositional representations? We use a novel analysis technique called ROLE to show t…
Using Priming to Uncover the Organization of Syntactic Representations in Neural Language Models
Grusha Prasad, Marten van Schijndel, Tal Linzen
Neural language models (LMs) perform well on tasks that require sensitivity to syntactic structure. Drawing on the syntactic priming paradigm from psycholinguistics, we propose a n…
Quantity doesn't buy quality syntax with neural language models
Marten van Schijndel, Aaron Mueller, Tal Linzen
Recurrent neural networks can learn to predict upcoming words remarkably well on average; in syntactically complex contexts, however, they often assign unexpectedly high probabilit…
Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshop
Afra Alishahi, Grzegorz Chrupała, Tal Linzen
The EMNLP 2018 workshop BlackboxNLP was dedicated to resources and techniques specifically developed for analyzing and understanding the inner-workings and representations acquired…
Probing What Different NLP Tasks Teach Machines about Function Word Comprehension
Najoung Kim, Roma Patel, Adam Poliak +9
We introduce a set of nine challenge tasks that test for the understanding of function words. These tasks are created by structurally mutating sentences from existing datasets to t…
Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages
Shauli Ravfogel, Yoav Goldberg, Tal Linzen
How do typological properties such as word order and morphological case marking affect the ability of neural sequence models to acquire the syntax of a language? Cross-linguistic c…
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
R. Thomas McCoy, Ellie Pavlick, Tal Linzen
A machine learning system can score well on a given test set by relying on heuristics that are effective for frequent example types but break down in more challenging cases. We stu…
Human few-shot learning of compositional instructions
Brenden M. Lake, Tal Linzen, Marco Baroni
People learn in fast and flexible ways that have not been emulated by machines. Once a person learns a new verb "dax," he or she can effortlessly understand how to "dax twice," "wa…