91 citations · 165 across the 33 of their papers we have counts for
29 papers · 1 filter
Why is AI-generated language so hard to understand?
Thomas Hikaru Clark, Matias B. Martin, Kyle Mahowald
There is a widespread sense that AI-generated text has recently become harder to understand. In this study, we investigate a set of psycholinguistic features that are known to corr…
Causal Interventions Reveal Typologically Organized Syntactic Mechanisms in Multilingual Language Models
Sasha Boguraev, Toshiki Nakai, Kyle Mahowald +1
Linguistic theory has long recognized cross-linguistic syntactic regularities, leading to claims that these similar structures are processed by similar mechanisms. However, this hy…
The Counterexample Game: Iterated Conceptual Analysis and Repair in Language Models
Daniel Drucker, Kyle Mahowald
Conceptual analysis -- proposing definitions and refining them through counterexamples -- is central to philosophical methodology. We study whether language models can perform this…
Causal Drawbridges: Characterizing Gradient Blocking of Syntactic Islands in Transformer LMs
Sasha Boguraev, Kyle Mahowald
We show how causal interventions in Transformer models provide insights into English syntax by focusing on a long-standing challenge for syntactic theory: syntactic islands. Extrac…
When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don't
Jonathan Nemitz, Carsten Eickhoff, Junyi Jessy Li +3
Understanding when Vision-Language Models (VLMs) will behave unexpectedly, whether models can reliably predict their own behavior, and if models adhere to their introspective reaso…
From sunblock to softblock: Analyzing the correlates of neology in published writing and on social media
Maria Ryskina, Matthew R. Gormley, Kyle Mahowald +3
Living languages are shaped by a host of conflicting internal and external evolutionary pressures. While some of these pressures are universal across languages and cultures, others…
LLMs and people both learn to form conventions -- just not with each other
Cameron R. Jones, Agnese Lombardi, Kyle Mahowald +1
Humans align to one another in conversation -- adopting shared conventions that ease communication. We test whether LLMs form the same kinds of conventions in a multimodal communic…
France or Spain or Germany or France: A Neural Account of Non-Redundant Redundant Disjunctions
Sasha Boguraev, Qing Yao, Kyle Mahowald
Sentences like "She will go to France or Spain, or perhaps to Germany or France." appear formally redundant, yet become acceptable in contexts such as "Mary will go to a philosophy…
What Can String Probability Tell Us About Grammaticality?
Jennifer Hu, Ethan Gotlieb Wilcox, Siyuan Song +2
What have language models (LMs) learned about grammar? This question remains hotly debated, with major ramifications for linguistic theory. However, since probability and grammatic…
Convergence and Divergence of Language Models under Different Random Seeds
Finlay Fehlauer, Kyle Mahowald, Tiago Pimentel
In this paper, we investigate the convergence of language models (LMs) trained under different random seeds, measuring convergence as the expected per-token Kullback--Leibler (KL)…
semantic-features: A User-Friendly Tool for Studying Contextual Word Embeddings in Interpretable Semantic Spaces
Jwalanthi Ranganathan, Rohan Jha, Kanishka Misra +1
We introduce semantic-features, an extensible, easy-to-use library based on Chronis et al. (2023) for studying contextualized word embeddings of LMs by projecting them into interpr…
Is It JUST Semantics? A Case Study of Discourse Particle Understanding in LLMs
William Sheffield, Kanishka Misra, Valentina Pyatkin +3
Discourse particles are crucial elements that subtly shape the meaning of text. These words, often polyfunctional, give rise to nuanced and often quite disparate semantic/discourse…
A suite of LMs comprehend puzzle statements as well as humans
Adele E Goldberg, Supantho Rakshit, Jennifer Hu +1
Recent claims suggest that large language models (LMs) underperform humans in comprehending minimally complex English statements (Dentella et al., 2024). Here, we revisit those fin…
Causal Interventions Reveal Shared Structure Across English Filler-Gap Constructions
Sasha Boguraev, Christopher Potts, Kyle Mahowald
Language Models (LMs) have emerged as powerful sources of evidence for linguists seeking to develop theories of syntax. In this paper, we argue that causal interpretability methods…
On Language Models' Sensitivity to Suspicious Coincidences
Sriram Padmanabhan, Kanishka Misra, Kyle Mahowald +1
Humans are sensitive to suspicious coincidences when generalizing inductively over data, as they make assumptions as to how the data was sampled. This results in smaller, more spec…
Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models
Qing Yao, Kanishka Misra, Leonie Weissweiler +1
Language models (LMs) tend to show human-like preferences on a number of syntactic phenomena, but the extent to which these are attributable to direct exposure to the phenomena or…
Language Models Fail to Introspect About Their Knowledge of Language
Siyuan Song, Jennifer Hu, Kyle Mahowald
There has been recent interest in whether large language models (LLMs) can introspect about their own internal states. Such abilities would make LLMs more interpretable, and also v…
Constructions are Revealed in Word Distributions
Joshua Rozner, Leonie Weissweiler, Kyle Mahowald +1
Construction grammar posits that constructions, or form-meaning pairings, are acquired through experience with language (the distributional learning hypothesis). But how much infor…
Linguistic Generalizations are not Rules: Impacts on Evaluation of LMs
Leonie Weissweiler, Kyle Mahowald, Adele Goldberg
Linguistic evaluations of how well LMs generalize to produce or understand language often implicitly take for granted that natural languages are generated by symbolic rules. Accord…
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez +7
Chain-of-thought (CoT) via prompting is the de facto method for eliciting reasoning capabilities from large language models (LLMs). But for what kinds of tasks is this extra ``thin…
Do they mean 'us'? Interpreting Referring Expressions in Intergroup Bias
Venkata S Govindarajan, Matianyu Zang, Kyle Mahowald +2
The variations between in-group and out-group speech (intergroup bias) are subtle and could underlie many social phenomena like stereotype perpetuation and implicit bias. In this p…
Participle-Prepended Nominals Have Lower Entropy Than Nominals Appended After the Participle
Kristie Denlinger, Stephen Wechsler, Kyle Mahowald
English allows for both compounds (e.g., London-made) and phrasal paraphrases (e.g., made in London). While these constructions have roughly the same truth-conditional meaning, we…
Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs
Kanishka Misra, Kyle Mahowald
Language models learn rare syntactic phenomena, but the extent to which this is attributable to generalization vs. memorization is a major open question. To that end, we iterativel…
Language models align with human judgments on key grammatical constructions
Jennifer Hu, Kyle Mahowald, Gary Lupyan +2
Do large language models (LLMs) make human-like linguistic generalizations? Dentella et al. (2023) ("DGL") prompt several LLMs ("Is the following sentence grammatically correct in…