18 citations · 52 across the 20 of their papers we have counts for
4 papers · 1 filter
Fabrication After Tool Failure: Tool-Augmented Agents Assert Values Their Tools Did Not Return
Arham Sethi, Arsen Kenzhebayev, Saanvi Paturi +3
Tool-augmented language models are evaluated on whether they reach the right answer, not on whether they report honestly when a tool fails to supply one. We isolate this post-failu…
When Tools Get in the Way: The Effect of Unnecessary Tool Availability on LLM Answering
Saanvi Paturi, Arsen Kenzhebayev, Arham Sethi +3
Large language models (LLMs) are increasingly deployed with external tools that extend what they can do beyond their own knowledge. Tools help on tasks that need external informati…
Knowing When Not to Answer: Pseudo-Ensembles for Abstention in Music Audio-Language Models
Aanya Maheshwari, Vatsal Raina
Music audio-language models are evaluated almost entirely by accuracy on multiple-choice questions. This protocol forces the model to commit to an option, so a lucky guess looks th…
Task Competence Is Not Instruction Following: Evaluating Instruction-Conflicting Behavior in Small Language Models
Mahdiyeh Farajidizaji, Vatsal Raina
Instruction tuning is meant to make language models follow user requests, yet it is unclear whether small models comply when an instruction conflicts with their usual task behavior…