76 citations · 120 across the 14 of their papers we have counts for
16 papers
Neural networks are more modular than single neurons suggest
Shreya Saha, Benjamin Bergen, Meenakshi Khosla
Neural networks routinely develop representations in which individual neurons contribute to multiple distinct functions, leading to the widespread view that such networks lack modu…
Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability
Tyler A. Chang, Zhuowen Tu, Benjamin K. Bergen
How do language models learn to make predictions during pre-training? To study this, we extract learning curves from five autoregressive English language model pre-training runs, f…
Emergent inabilities? Inverse scaling over the course of pretraining
James A. Michaelov, Benjamin K. Bergen
Does inverse scaling only occur as a function of model size, or can it also occur over the course of training? We carry out an exploratory study investigating whether the performan…
Language Model Behavior: A Comprehensive Survey
Tyler A. Chang, Benjamin K. Bergen
Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers. In this survey, we discuss over 250 rec…
Can Peanuts Fall in Love with Distributional Semantics?
James A. Michaelov, Seana Coulson, Benjamin K. Bergen
Context changes expectations about upcoming words - following a story involving an anthropomorphic peanut, comprehenders expect the sentence the peanut was in love more than the pe…
Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers
James A. Michaelov, Benjamin K. Bergen
How well do language models deal with quantification? In this study, we focus on 'few'-type quantifiers, as in 'few children like toys', which might pose a particular challenge for…
Collateral facilitation in humans and language models
James A. Michaelov, Benjamin K. Bergen
Are the predictions of humans and language models affected by similar things? Research suggests that while comprehending language, humans make predictions about upcoming words, wit…
Do Large Language Models know what humans know?
Sean Trott, Cameron Jones, Tyler Chang +2
Humans can attribute beliefs to others. However, it is unknown to what extent this ability results from an innate biological endowment or from experience accrued through child deve…
Do language models make human-like predictions about the coreferents of Italian anaphoric zero pronouns?
James A. Michaelov, Benjamin K. Bergen
Some languages allow arguments to be omitted in certain contexts. Yet human language comprehenders reliably infer the intended referents of these zero pronouns, in part because the…
The Geometry of Multilingual Language Model Representations
Tyler A. Chang, Zhuowen Tu, Benjamin K. Bergen
We assess how multilingual language models maintain a shared multilingual representation space while still encoding language-sensitive information in each language. Using XLM-R as…
Contextualized Sensorimotor Norms: multi-dimensional measures of sensorimotor strength for ambiguous English words, in context
Sean Trott, Benjamin Bergen
Most large language models are trained on linguistic input alone, yet humans appear to ground their understanding of words in sensorimotor experience. A natural solution is to augm…
Word Acquisition in Neural Language Models
Tyler A. Chang, Benjamin K. Bergen
We investigate how neural language models acquire individual words during training, extracting learning curves and ages of acquisition for over 600 words on the MacArthur-Bates Com…
So Cloze yet so Far: N400 Amplitude is Better Predicted by Distributional Information than Human Predictability Judgements
James A. Michaelov, Seana Coulson, Benjamin K. Bergen
More predictable words are easier to process - they are read faster and elicit smaller neural signals associated with processing difficulty, most notably, the N400 component of the…
Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude?
James A. Michaelov, Megan D. Bardolph, Seana Coulson +1
Despite being designed for performance rather than cognitive plausibility, transformer language models have been found to be better at predicting metrics used to assess human langu…
RAW-C: Relatedness of Ambiguous Words--in Context (A New Lexical Resource for English)
Sean Trott, Benjamin Bergen
Most words are ambiguous--i.e., they convey distinct meanings in different contexts--and even the meanings of unambiguous words are context-dependent. Both phenomena present a chal…
How well does surprisal explain N400 amplitude under different experimental conditions?
James A. Michaelov, Benjamin K. Bergen
We investigate the extent to which word surprisal can be used to predict a neural measure of human language processing difficulty - the N400. To do this, we use recurrent neural ne…