146 citations · 155 across the 7 of their papers we have counts for
9 papers · 1 filter
A Cross-Linguistic Pressure for Uniform Information Density in Word Order
Thomas Hikaru Clark, Clara Meister, Tiago Pimentel +4
While natural languages differ widely in both canonical word order and word order flexibility, their word orders still follow shared cross-linguistic statistical patterns, often at…
A Theory of Emergent In-Context Learning as Implicit Structure Induction
Michael Hahn, Navin Goyal
Scaling large language models (LLMs) leads to an emergent capacity to learn in-context from example demonstrations. Despite progress, theoretical understanding of this phenomenon r…
Sensitivity as a Complexity Measure for Sequence Classification Tasks
Michael Hahn, Dan Jurafsky, Richard Futrell
We introduce a theoretical framework for understanding and predicting the complexity of sequence classification tasks, using a novel extension of the theory of Boolean function sen…
RNNs can generate bounded hierarchical languages with optimal memory
John Hewitt, Michael Hahn, Surya Ganguli +2
Recurrent neural networks empirically generate natural language with high syntactic fidelity. However, their success is not well-understood theoretically. We provide theoretical in…
Tabula nearly rasa: Probing the Linguistic Knowledge of Character-Level Neural Language Models Trained on Unsegmented Text
Michael Hahn, Marco Baroni
Recurrent neural networks (RNNs) have reached striking performance in many natural language processing tasks. This has renewed interest in whether these generic sequence processing…
Theoretical Limitations of Self-Attention in Neural Sequence Models
Michael Hahn
Transformers are emerging as the new workhorse of NLP, showing great success across tasks. Unlike LSTMs, transformers process input sequences entirely through self-attention. Previ…
Character-based Surprisal as a Model of Reading Difficulty in the Presence of Error
Michael Hahn, Frank Keller, Yonatan Bisk +1
Intuitively, human readers cope easily with errors in text; typos, misspelling, word substitutions, etc. do not unduly disrupt natural reading. Previous work indicates that letter…
Modeling Task Effects in Human Reading with Neural Network-based Attention
Michael Hahn, Frank Keller
Research on human reading has long documented that reading behavior shows task-specific effects, but it has been challenging to build general models predicting what reading behavio…
Modeling Human Reading with Neural Attention
Michael Hahn, Frank Keller
When humans read text, they fixate some words and skip others. However, there have been few attempts to explain skipping behavior with computational models, as most existing work h…