11 citations · 20 across the 10 of their papers we have counts for
4 papers · 1 filter
Evaluation Awareness Is Not One Capability: Evidence from Open Language Models
Nilesh Nayan, Aishwarya Sampath Kumar, Rishiraj Girmal +5
Safety benchmarks assume that test-condition behavior predicts deployment behavior, an assumption that fails if models detect evaluation cues and adapt. This opens a gap between be…
Detecting and Characterizing Planning in Language Models
Jatin Nainani, Sankaran Vaidyanathan, Connor Watts +2
Modern large language models (LLMs) have demonstrated impressive performance across a wide range of multi-step reasoning tasks. Recent work suggests that LLMs may perform planning…
Quantitative LLM Judges
Aishwarya Sahoo, Jeevana Kruthi Karnuthala, Tushar Parmanand Budhwani +9
LLM-as-a-judge is a framework where a large language model (LLM) evaluates the output of another LLM. While LLMs excel at producing qualitative textual evaluations, they often stru…
Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
Aman Singh Thakur, Kartik Choudhary, Venkat Srinik Ramayapally +2
Offering a promising solution to the scalability challenges associated with human evaluation, the LLM-as-a-judge paradigm is rapidly gaining traction as an approach to evaluating l…