8 papers · 1 filter
Sycophancy Undermines Epistemic Vigilance in Cooperative Vision-Language Tasks
Rupak Sarkar, Neha Srikanth, Saloni Gupta +3
To maintain common ground in cooperative conversation, humans iteratively update their beliefs as conversation participants share new information; participants who are epistemicall…
Linguistic Productivity in Large Language Models: Models Coerce, but do not Preempt
Claire Bonial, Claire Benet Post, Laura Michaelis +1
Usage-based theories of grammars posit that creative productivity of the structures of language is both bolstered and constrained by two distinct frequency signals: entrenchment, s…
Beyond Memorization: Assessing Semantic Generalization in Large Language Models Using Phrasal Constructions
Wesley Scivetti, Melissa Torgbi, Austin Blodgett +4
The web-scale of pretraining data has created an important evaluation challenge: to disentangle linguistic competence on cases well-represented in pretraining data from generalizat…
Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priors
Harish Tayyar Madabushi, Melissa Torgbi, Claire Bonial
In this position paper we raise critical awareness of a realistic view of LLM capabilities that eschews extreme alternative views that LLMs are either 'stochastic parrots' or in po…
Evaluating CxG Generalisation in LLMs via Construction-Based NLI Fine Tuning
Tom Mackintosh, Harish Tayyar Madabushi, Claire Bonial
We probe large language models' ability to learn deep form-meaning mappings as defined by construction grammars. We introduce the ConTest-NLI benchmark of 80k sentences covering ei…
FRIDA to the Rescue! Analyzing Synthetic Data Effectiveness in Object-Based Common Sense Reasoning for Disaster Response
Mollie Shichman, Claire Bonial, Austin Blodgett +3
During Human Robot Interactions in disaster relief scenarios, Large Language Models (LLMs) have the potential for substantial physical reasoning to assist in mission objectives. Ho…