11 papers
Sycophancy Undermines Epistemic Vigilance in Cooperative Vision-Language Tasks
Rupak Sarkar, Neha Srikanth, Saloni Gupta +3
To maintain common ground in cooperative conversation, humans iteratively update their beliefs as conversation participants share new information; participants who are epistemicall…
Linguistic Productivity in Large Language Models: Models Coerce, but do not Preempt
Claire Bonial, Claire Benet Post, Laura Michaelis +1
Usage-based theories of grammars posit that creative productivity of the structures of language is both bolstered and constrained by two distinct frequency signals: entrenchment, s…
Beyond Memorization: Assessing Semantic Generalization in Large Language Models Using Phrasal Constructions
Wesley Scivetti, Melissa Torgbi, Austin Blodgett +4
The web-scale of pretraining data has created an important evaluation challenge: to disentangle linguistic competence on cases well-represented in pretraining data from generalizat…
Petri Net Relaxation for Infeasibility Explanation and Sequential Task Planning
Nguyen Cong Nhat Le, John G. Rogers, Claire N. Bonial +1
Plans often change due to changes in the situation or our understanding of the situation. Sometimes, a feasible plan may not even exist, and identifying such infeasibilities is use…
Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priors
Harish Tayyar Madabushi, Melissa Torgbi, Claire Bonial
In this position paper we raise critical awareness of a realistic view of LLM capabilities that eschews extreme alternative views that LLMs are either 'stochastic parrots' or in po…
Evaluating CxG Generalisation in LLMs via Construction-Based NLI Fine Tuning
Tom Mackintosh, Harish Tayyar Madabushi, Claire Bonial
We probe large language models' ability to learn deep form-meaning mappings as defined by construction grammars. We introduce the ConTest-NLI benchmark of 80k sentences covering ei…