1 citations · 1 across the 5 of their papers we have counts for
5 papers
Interactive Evaluation Requires a Design Science
Keyang Xuan, Peiyang Song, Pan Lu +10
AI evaluation is undergoing a structural change. Large language models (LLMs) are increasingly deployed as systems that act over time through tools, environments, users, and other…
Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges
Ruomeng Ding, Yifei Pang, He Sun +3
Evaluation and alignment pipelines for large language models increasingly rely on LLM-based judges, whose behavior is guided by natural-language rubrics and validated on benchmarks…
EvalTree: Profiling Language Model Weaknesses via Hierarchical Capability Trees
Zhiyuan Zeng, Yizhong Wang, Hannaneh Hajishirzi +1
An ideal model evaluation should achieve two goals: identifying where the model fails and providing actionable improvement guidance. Toward these goals for language model (LM) eval…
Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedback
Lester James V. Miranda, Yizhong Wang, Yanai Elazar +6
Learning from human feedback has enabled the alignment of language models (LMs) with human preferences. However, collecting human preferences is expensive and time-consuming, with…
Third-Party Language Model Performance Prediction from Instruction
Rahul Nadkarni, Yizhong Wang, Noah A. Smith
Language model-based instruction-following systems have lately shown increasing performance on many benchmark tasks, demonstrating the capability of adapting to a broad variety of…