63 citations · 110 across the 16 of their papers we have counts for
17 papers
False perspectives on human language: why statistics needs linguistics
Matteo Greco, Andrea Cometa, Fiorenzo Artoni +2
A sharp tension exists about the nature of human language between two opposite parties: those who believe that statistical surface distributions, in particular using measures like…
How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech
Aditya Yedetore, Tal Linzen, Robert Frank +1
When acquiring syntax, children consistently choose hierarchical rules over competing non-hierarchical possibilities. Is this preference due to a learning bias for hierarchical str…
Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity
Yiding Hao, Dana Angluin, Robert Frank
This paper analyzes three formal models of Transformer encoders that differ in the form of their self-attention mechanism: unique hard attention (UHAT); generalized unique hard att…
Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Models
Aaron Mueller, Robert Frank, Tal Linzen +2
Relations between words are governed by hierarchical structure rather than linear ordering. Sequence-to-sequence (seq2seq) models, despite their success in downstream NLP applicati…
Do Language Models Learn Position-Role Mappings?
Jackson Petty, Michael Wilson, Robert Frank
How is knowledge of position-role mappings in natural language learned? We explore this question in a computational setting, testing whether a variety of well-performing pertained…
Transformers Generalize Linearly
Jackson Petty, Robert Frank
Natural language exhibits patterns of hierarchically governed dependencies, in which relations between words are sensitive to syntactic structure rather than linear ordering. While…