139 citations · 462 across the 39 of their papers we have counts for
5 papers · 1 filter
Picking BERT's Brain: Probing for Linguistic Dependencies in Contextualized Embeddings Using Representational Similarity Analysis
Michael A. Lepori, R. Thomas McCoy
As the name implies, contextualized representations of language are typically motivated by their ability to encode context. Which aspects of context are captured by such representa…
Universal linguistic inductive biases via meta-learning
R. Thomas McCoy, Erin Grant, Paul Smolensky +2
How do learners acquire languages from the limited data available to them? This process must involve some inductive biases - factors that affect how a learner generalizes - but it…
Representations of Syntax [MASK] Useful: Effects of Constituency and Dependency Structure in Recursive LSTMs
Michael A. Lepori, Tal Linzen, R. Thomas McCoy
Sequence-based neural networks show significant sensitivity to syntactic structure, but they still perform less well on syntactic tasks than tree-based networks. Such tree-based ne…
Syntactic Data Augmentation Increases Robustness to Inference Heuristics
Junghyun Min, R. Thomas McCoy, Dipanjan Das +2
Pretrained neural models such as BERT, when fine-tuned to perform natural language inference (NLI), often show high accuracy on standard datasets, but display a surprising lack of…
Does syntax need to grow on trees? Sources of hierarchical inductive bias in sequence-to-sequence networks
R. Thomas McCoy, Robert Frank, Tal Linzen
Learners that are exposed to the same training data might generalize differently due to differing inductive biases. In neural network models, inductive biases could in theory arise…