4 papers
The Wording Effect: Quantifying Two-Way Drift in LLM Benchmark Performance
Shailja Thakur, Sungeun An, Chad DeLuca +1
A benchmark score comes from a single phrasing of each problem. That single phrasing is treated as if it stood for the whole space of ways the same problem could be asked, but it d…
A Systematic Approach for Large Language Models Debugging
Basel Shbita, Anna Lisa Gentile, Bing Zhang +10
Large language models (LLMs) have become central to modern AI workflows, powering applications from open-ended text generation to complex agent-based reasoning. However, debugging…
NC-Bench: An LLM Benchmark for Evaluating Conversational Competence
Robert J. Moore, Sungeun An, Farhan Ahmed +1
Existing LLM benchmarks evaluate what models say, such as whether answers are correct, faithful, or helpful, but they do not test whether models produce the right type of conversat…
Data-Prep-Kit: getting your data ready for LLM application development
David Wood, Boris Lublinsky, Alexy Roytman +21
Data preparation is the first and a very important step towards any Large Language Model (LLM) development. This paper introduces an easy-to-use, extensible, and scale-flexible ope…