5 papers
StatABench: Dataset and Framework for Evaluating Statistical Analysis Capabilities of LLMs
Youxin Zhu, Yixuan Ding, Peng Lai +3
Statistical analysis is a broad, complex field requiring both domain knowledge and tool proficiency. While prior work has evaluated large language models (LLMs) in this domain, exi…
Bridging the Agent-World Gap: Text World Models for LLM-based Agents
Yixia Li, Hongru Wang, Peng Lai +13
Large language model (LLM)-based agents are increasingly used in interactive textual environments, from web navigation and code editing to tool use and long-horizon dialogue. Yet m…
BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation
Peng Lai, Zhihao Ou, Yong Wang +4
LLM-as-a-Judge has been widely adopted across various research and practical applications, yet the robustness and reliability of its evaluation remain a critical issue. A core chal…
Anchored Supervised Fine-Tuning
He Zhu, Junyou Su, Peng Lai +4
Post-training of large language models involves a fundamental trade-off between supervised fine-tuning (SFT), which efficiently mimics demonstrations but tends to memorize, and rei…
Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations
Peng Lai, Jianjie Zheng, Sijie Cheng +4
The growing scale of evaluation tasks has led to the widespread adoption of automated evaluation using LLMs, a paradigm known as "LLM-as-a-judge". However, improving its alignment…