5 papers
The Emergence and Sparsification of a Syntactic Circuit
Qing Yao, Sasha Boguraev, Tiago Pimentel +1
To understand the emergence of grammatical behavior in language models, we track circuits for subject-verb number agreement in pythia-1b and ten seeds of pythia-410m across trainin…
What's in a Name? Morphological Shortcuts by LLMs in Pharmacology
Kaijie Mo, Thomas Yang, Chantal Shaib +6
The morphological form of a word can often give cues to its meaning, but purely relying on these mappings can lead to overgeneralization in high-stakes domains. In the medical doma…
France or Spain or Germany or France: A Neural Account of Non-Redundant Redundant Disjunctions
Sasha Boguraev, Qing Yao, Kyle Mahowald
Sentences like "She will go to France or Spain, or perhaps to Germany or France." appear formally redundant, yet become acceptable in contexts such as "Mary will go to a philosophy…
Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models
Qing Yao, Kanishka Misra, Leonie Weissweiler +1
Language models (LMs) tend to show human-like preferences on a number of syntactic phenomena, but the extent to which these are attributable to direct exposure to the phenomena or…
Random space-time sampling and reconstruction of sparse bandlimited graph diffusion field
Longxiu Huang, Dongyang Li, Sui Tang +1
In this work, we investigate the sampling and reconstruction of spectrally -sparse bandlimited graph signals governed by heat diffusion processes. We propose a random space-time…