7 papers
Winning by Peeking: Unenforced Budgets and Test-Set Selection Inflate Short-Budget AutoML Comparisons
Guilin Zhang, Kai Zhao
Comparisons between AutoML systems at short time budgets -- tens of seconds rather than hours -- are common in tool READMEs and workshop papers, and they are easy to get wrong. We…
Deployment-Time Memorization in Foundation-Model Agents
Lei, Chen, Guilin Zhang +8
Foundation-model agents are increasingly long-lived systems that remember users across interactions, making memorization an explicit deployment-time function rather than solely a p…
EvalStop: Using World Feedback to Detect and Correct Reward Overoptimization in Multi-Tenant RLHF Platforms
Guilin Zhang, Chuanyi Sun, Kai Zhao +3
Cloud LLM fine-tuning platforms increasingly serve RLHF workloads, where a learned reward model is optimized as a proxy for human quality. As Gao et al. (2023) showed, this proxy d…
Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation
Xue Liu, Xin Ma, Yuxin Ma +36
As Large Language Models (LLMs) exhibit plateauing performance on conventional benchmarks, a pivotal challenge persists: evaluating their proficiency in complex, open-ended tasks c…
Adaptive Memory Admission Control for LLM Agents
Guilin Zhang, Wei Jiang, Xiejiashan Wang +5
LLM-based agents increasingly rely on long-term memory to support multi-session reasoning and interaction, yet current systems provide little control over what information is retai…
LLMs as Orchestrators: Constraint-Compliant Multi-Agent Optimization for Recommendation Systems
Guilin Zhang, Kai Zhao, Jeffrey Friedman +1
Recommendation systems must optimize multiple objectives while satisfying hard business constraints such as fairness and coverage. For example, an e-commerce platform may require e…