4 papers
Learning from Less: Measuring the Effectiveness of RLVR in Low Data and Compute Regimes
Justin Bauer, Thomas Walshe, Derek Pham +4
Fine-tuning Large Language Models (LLMs) typically relies on large quantities of high-quality annotated data, or questions with well-defined ground truth answers in the case of Rei…
Automating Benchmark Design
Amanda Dsouza, Harit Vishwakarma, Zhengyang Qi +6
The rapid progress and widespread deployment of LLMs and LLM-powered agents has outpaced our ability to evaluate them. Hand-crafted, static benchmarks are the primary tool for asse…
Pearls from Pebbles: Improved Confidence Functions for Auto-labeling
Harit Vishwakarma, Reid, Chen +4
Auto-labeling is an important family of techniques that produce labeled training sets with minimum manual labeling. A prominent variant, threshold-based auto-labeling (TBAL), works…
The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models
Satya Sai Srinath Namburi, Makesh Sreedhar, Srinath Srinivasan +1
Compressing large language models (LLMs), often consisting of billions of parameters, provides faster inference, smaller memory footprints, and enables local deployment. Two standa…