10 citations · 11 across the 9 of their papers we have counts for
9 papers
The Wording Effect: Quantifying Two-Way Drift in LLM Benchmark Performance
Shailja Thakur, Sungeun An, Chad DeLuca +1
A benchmark score comes from a single phrasing of each problem. That single phrasing is treated as if it stood for the whole space of ways the same problem could be asked, but it d…
A Systematic Approach for Large Language Models Debugging
Basel Shbita, Anna Lisa Gentile, Bing Zhang +10
Large language models (LLMs) have become central to modern AI workflows, powering applications from open-ended text generation to complex agent-based reasoning. However, debugging…
STaD: Scaffolded Task Design for Identifying Compositional Skill Gaps in LLMs
Sungeun An, Swanand Ravindra Kadhe, Shailja Thakur +2
Benchmarks are often used as a standard to understand LLM capabilities in different domains. However, aggregate benchmark scores provide limited insight into compositional skill ga…
CIFE: Code Instruction-Following Evaluation
Sravani Gunnu, Shanmukha Guttula, Hima Patel
Large Language Models (LLMs) are increasingly applied to real-world code generation, where functional correctness alone is insufficient for reliable deployment, developers also exp…
STRIDE: A Systematic Framework for Selecting AI Modalities -- Agentic AI, AI Assistants, or LLM Calls
Shubhi Asthana, Bing Zhang, Chad DeLuca +2
The rapid shift from stateless large language models (LLMs) to autonomous, goal-driven agents raises a central question: When is agentic AI truly necessary? While agents enable mul…
Generating Verifiable Chain of Thoughts from Exection-Traces
Shailja Thakur, Vaibhav Saxena, Rohan Kulkarni +4
Getting language models to reason correctly about code requires training on data where each reasoning step can be checked. Current synthetic Chain-of-Thought (CoT) training data of…
Data-Prep-Kit: getting your data ready for LLM application development
David Wood, Boris Lublinsky, Alexy Roytman +21
Data preparation is the first and a very important step towards any Large Language Model (LLM) development. This paper introduces an easy-to-use, extensible, and scale-flexible ope…
Scaling Granite Code Models to 128K Context
Matt Stallone, Vaibhav Saxena, Leonid Karlinsky +19
This paper introduces long-context Granite code models that support effective context windows of up to 128K tokens. Our solution for scaling context length of Granite 3B/8B code mo…
Granite Code Models: A Family of Open Foundation Models for Code Intelligence
Mayank Mishra, Matt Stallone, Gaoyuan Zhang +43
Large Language Models (LLMs) trained on code are revolutionizing the software development process. Increasingly, code LLMs are being integrated into software development environmen…