72 citations · 77 across the 17 of their papers we have counts for
19 papers
Using Prosody to Predict Syntactic Structure
Junghyun Min, Alex Warstadt, Tamar I. Regev +2
While it is well-established that prosody carries crucial cues for syntactic structure, the degree and nature of correspondence between these two domains remains contested. We inve…
Beyond Static and Linear: What Attention Constraints Best Fit Human Reading Times?
Lanni Bu, Xiulin Yang, Christian Clark +2
Transformer-based language models are widely used as models of human language processing, yet their attention mechanisms allow lossless access to the full preceding context, unlike…
Dual Alignment Between Language Model Layers and Human Sentence Processing
Tatsuki Kuribayashi, Alex Warstadt, Yohei Oseki +1
A recent study (Kuribayashi et al., 2025) has shown that human sentence processing behavior, typically measured on syntactically unchallenging constructions, can be effectively mod…
BabyLM Turns 4 and Goes Multilingual: Call for Papers for the 2026 BabyLM Workshop
Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul +7
The goal of the BabyLM is to stimulate new research connections between cognitive modeling and language model pretraining. We invite contributions in this vein to the BabyLM Worksh…
What Do Prosody and Text Convey? Characterizing How Meaningful Information is Distributed Across Multiple Channels
Aditya Yadavalli, Tiago Pimentel, Tamar I Regev +2
Prosody -- the melody of speech -- conveys critical information often not captured by the words or text of a message. In this paper, we propose an information-theoretic approach to…
Modeling Bottom-up Information Quality during Language Processing
Cui Ding, Yanning Yin, Lena A. Jäger +1
Contemporary theories model language processing as integrating both top-down expectations and bottom-up inputs. One major prediction of such models is that the quality of the botto…
The Harmonic Structure of Information Contours
Eleftheria Tsipidi, Samuel Kiegeland, Franz Nowak +5
The uniform information density (UID) hypothesis proposes that speakers aim to distribute information evenly throughout a text, balancing production effort and listener comprehensi…
Using Information Theory to Characterize Prosodic Typology: The Case of Tone, Pitch-Accent and Stress-Accent
Ethan Gotlieb Wilcox, Cui Ding, Giovanni Acampa +3
This paper argues that the relationship between lexical identity and prosody -- one well-studied parameter of linguistic variation -- can be characterized using information theory.…
Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
Alex Warstadt, Aaron Mueller, Leshem Choshen +8
Children can acquire language from less than 100 million words of input. Large language models are far less data-efficient: they typically require 3 or 4 orders of magnitude more d…
The time scale of redundancy between prosody and linguistic context
Tamar I. Regev, Chiebuka Ohams, Shaylee Xie +5
In spoken communication, information is transmitted not only via words, but also through a rich array of non-verbal signals, including prosody--the non-segmental auditory features…
BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop
Lucas Charpentier, Leshem Choshen, Ryan Cotterell +11
BabyLM aims to dissolve the boundaries between cognitive modeling and language modeling. We call for both workshop papers and for researchers to join the 3rd BabyLM competition. As…
Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
Michael Y. Hu, Aaron Mueller, Candace Ross +7
The BabyLM Challenge is a community effort to close the data-efficiency gap between human and computational language learners. Participants compete to optimize language model train…
Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discourse
Eleftheria Tsipidi, Franz Nowak, Ryan Cotterell +3
The Uniform Information Density (UID) hypothesis posits that speakers tend to distribute information evenly across linguistic units to achieve efficient communication. Of course, i…
Reverse-Engineering the Reader
Samuel Kiegeland, Ethan Gotlieb Wilcox, Afra Amini +2
Numerous previous studies have sought to determine to what extent language models, pretrained on natural language text, can serve as useful models of human cognition. In this paper…
On the Role of Context in Reading Time Prediction
Andreas Opedal, Eleanor Chodroff, Ryan Cotterell +1
We present a new perspective on how readers integrate context during real-time language comprehension. Our proposals build on surprisal theory, which posits that the processing eff…
[Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus
Leshem Choshen, Ryan Cotterell, Michael Y. Hu +7
After last year's successful BabyLM Challenge, the competition will be hosted again in 2024/2025. The overarching goals of the challenge remain the same; however, some of the compe…
WhisBERT: Multimodal Text-Audio Language Modeling on 100M Words
Lukas Wolf, Greta Tuckute, Klemen Kotar +4
Training on multiple modalities of input can augment the capabilities of a language model. Here, we ask whether such a training regime can improve the quality and efficiency of the…
Revisiting the Optimality of Word Lengths
Tiago Pimentel, Clara Meister, Ethan Gotlieb Wilcox +2
Zipf (1935) posited that wordforms are optimized to minimize utterances' communicative costs. Under the assumption that cost is given by an utterance's length, he supported this cl…
Quantifying the redundancy between prosody and text
Lukas Wolf, Tiago Pimentel, Evelina Fedorenko +4
Prosody -- the suprasegmental component of speech, including pitch, loudness, and tempo -- carries critical aspects of meaning. However, the relationship between the information co…