11 papers
SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinement
Srishti Gautam, Arjun Radhakrishna, Sumit Gulwani
Skill documents, structured natural-language instructions that guide Large Language Model (LLM) agents, are critical to modern agent frameworks, yet LLMs struggle to write skills t…
An Empirical Investigation of Robustness in Large Language Models under Tabular Distortions
Avik Dutta, Harshit Nigam, Hosein Hasanbeig +2
We investigate how large language models (LLMs) fail when tabular data in an otherwise canonical representation is subjected to semantic and structural distortions. Our findings re…
Scaling Competence, Shrinking Reasoning: Cognitive Signatures in Language Model Learning
Mukul Singh, Ananya Singha, Arjun Radhakrishna +1
We analyze reasoning in language models during task-specific fine-tuning and draws parallel between reasoning tokens--intermediate steps generated while solving problem and the hum…
STACKFEED: Structured Textual Actor-Critic Knowledge Base Editing with FeedBack
Shashank Kirtania, Naman Gupta, Priyanshu Gupta +7
Large Language Models (LLMs) often generate incorrect or outdated information, especially in low-resource settings or when dealing with private data. To address this, Retrieval-Aug…
ConDABench: Interactive Evaluation of Language Models for Data Analysis
Avik Dutta, Priyanshu Gupta, Hosein Hasanbeig +6
Real-world data analysis tasks often come with under-specified goals and unclean data. User interaction is necessary to understand and disambiguate a user's intent, and hence, esse…
Do Code Models Suffer from the Dunning-Kruger Effect?
Mukul Singh, Somya Chatterjee, Arjun Radhakrishna +1
As artificial intelligence systems increasingly collaborate with humans in creative and technical domains, questions arise about the cognitive boundaries and biases that shape our…