6 papers
Similarity All The Way Up: Multilingual Generalization in LLMs Relies on Language-Level Similarity Structures
Supantho Rakshit, Adele Goldberg, Henry Conklin
As Large Language Models (LLMs) grow more capable across diverse tasks, their (in)ability to generalize remains difficult to quantify and poorly understood beyond limited domains.…
Large Language Models provide support for the parallelogram theory of analogy
Qiawen Ella Liu, Raja Marjieh, Jian-Qiao Zhu +2
Four-term word analogies (A:B::C:D) are classically modeled geometrically as parallelograms: adding the vector B-A+C produces D. Recent work suggests that this model poorly capture…
Linguistic Generalizations are not Rules: Impacts on Evaluation of LMs
Leonie Weissweiler, Kyle Mahowald, Adele Goldberg
Linguistic evaluations of how well LMs generalize to produce or understand language often implicitly take for granted that natural languages are generated by symbolic rules. Accord…
Meaning-infused grammar: Gradient Acceptability Shapes the Geometric Representations of Constructions in LLMs
Supantho Rakshit, Adele Goldberg
The usage-based constructionist (UCx) approach to language posits that language comprises a network of learned form-meaning pairings (constructions) whose use is largely determined…
For GPT-4 as with Humans: Information Structure Predicts Acceptability of Long-Distance Dependencies
Nicole Cuneo, Eleanor Graves, Supantho Rakshit +1
It remains debated how well any LM understands natural language or generates reliable metalinguistic judgments. Moreover, relatively little work has demonstrated that LMs can repre…
A suite of LMs comprehend puzzle statements as well as humans
Adele E Goldberg, Supantho Rakshit, Jennifer Hu +1
Recent claims suggest that large language models (LMs) underperform humans in comprehending minimally complex English statements (Dentella et al., 2024). Here, we revisit those fin…