1 citations · 1 across the 12 of their papers we have counts for
7 papers · 1 filter
ChartRevise: A Dataset and Evaluation Protocol for Exact Chart Editing via Code
Jiaxiang Tang, Yi Zhou, Chad DeLuca +5
Chart editing requires cross-modal edit grounding, realizing a requested visual change in the code that draws it, with necessary related updates and without altering unrelated cont…
The Wording Effect: Quantifying Two-Way Drift in LLM Benchmark Performance
Shailja Thakur, Sungeun An, Chad DeLuca +1
A benchmark score comes from a single phrasing of each problem. That single phrasing is treated as if it stood for the whole space of ways the same problem could be asked, but it d…
Runtime-Structured Task Decomposition for Agentic Coding Systems
Shubhi Asthana, Bing Zhang, Chad DeLuca +2
Agentic coding systems increasingly use large language models (LLMs) for software engineering tasks such as debugging, root cause analysis, and code review. However, many existing…
A Systematic Approach for Large Language Models Debugging
Basel Shbita, Anna Lisa Gentile, Bing Zhang +10
Large language models (LLMs) have become central to modern AI workflows, powering applications from open-ended text generation to complex agent-based reasoning. However, debugging…
STaD: Scaffolded Task Design for Identifying Compositional Skill Gaps in LLMs
Sungeun An, Swanand Ravindra Kadhe, Shailja Thakur +2
Benchmarks are often used as a standard to understand LLM capabilities in different domains. However, aggregate benchmark scores provide limited insight into compositional skill ga…
LLMON: An LLM-native Markup Language to Leverage Structure and Semantics at the LLM Interface
Michael Hind, Basel Shbita, Bo Wu +5
Textual Large Language Models (LLMs) provide a simple and familiar interface: a string of text is used for both input and output. However, the information conveyed to an LLM often…
LogitScope: A Framework for Analyzing LLM Uncertainty Through Information Metrics
Farhan Ahmed, Yuya Jeremy Ong, Chad DeLuca
Understanding and quantifying uncertainty in large language model (LLM) outputs is critical for reliable deployment. However, traditional evaluation approaches provide limited insi…