10 papers
An Emergent Mirage: Is Emergent Misalignment and Realignment Indeed a Robust Phenomenon?
Abhinav Rao, Liancheng Gong, Bin Hu +1
Recent work has reported Emergent Misalignment (EM), where language models fine-tuned on narrow, domain-specific misaligned datasets abruptly acquire broadly misaligned behavior, a…
Steer, Don't Solve: Training Small Critic Models for Large Code Agents
Shubham Gandhi, Yiqing Xie, Atharva Naik +2
End-to-end code agent training is resource-intensive and plateaus on the strategy-level reasoning needed to resolve code issues, since jointly optimizing code-level execution and s…
ReaComp: Compiling LLM Reasoning into Symbolic Solvers for Efficient Program Synthesis
Atharva Naik, Yash Mathur, Prakam +2
LLMs can solve program synthesis tasks but remain inefficient and unreliable on hard instances requiring large combinatorial search. Given a small set of reasoning traces, we use c…
MetaLint: Easy-to-Hard Generalization for Code Linting
Atharva Naik, Lawanya Baghel, Dhakshin Govindarajan +4
Large language models excel at code generation but struggle with code linting, particularly in generalizing to unseen or evolving best practices beyond those observed during traini…
ChartEditBench: Evaluating Grounded Multi-Turn Chart Editing in Multimodal Language Models
Manav Nitin Kapadnis, Lawanya Baghel, Atharva Naik +1
While Multimodal Large Language Models (MLLMs) perform strongly on single-turn chart generation, their ability to support real-world exploratory data analysis remains underexplored…
PBEBench: A Multi-Step Programming by Examples Reasoning Benchmark inspired by Historical Linguistics
Atharva Naik, Prakam, Yash Mathur +6
Although many benchmarks evaluate the reasoning abilities of Large Language Models (LLMs) within domains such as mathematics, coding, or data wrangling, few abstract away from doma…