1 citations · 1 across the 6 of their papers we have counts for
12 papers
AcademiClaw: When Students Set Challenges for AI Agents
Junjie Yu, Pengrui Lu, Weiye Si +75
Benchmarks within the OpenClaw ecosystem have thus far evaluated exclusively assistant-level tasks, leaving the academic-level capabilities of OpenClaw largely unexamined. We intro…
AlphaEval: Evaluating Agents in Production
Pengrui Lu, Bingyu Xu, Wenjun Zhang +24
The rapid deployment of AI agents in commercial settings has outpaced the development of evaluation methodologies that reflect production realities. Existing benchmarks measure age…
NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions
Shizheng Hou, Wenqi Pei, Nuo Chen +3
Natural Language to SQL (NL2SQL) technology empowers non-expert users to query relational databases without requiring SQL expertise. While large language models (LLMs) have greatly…
Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Tasks
Tianze Xu, Yanzhao Zheng, Pengrui Lu +11
Rubric-based Reinforcement Learning (RL) has emerged as a promising approach for aligning Large Language Models (LLMs) with complex, open-domain instruction following tasks. Howeve…
daVinci-LLM:Towards the Science of Pretraining
Yiwei Qin, Yixiu Liu, Tiantian Mi +12
The foundational pretraining phase determines a model's capability ceiling, as post-training struggles to overcome capability foundations established during pretraining, yet it rem…
ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development
Pengrui Lu, Shiqi Zhang, Yunzhong Hou +8
Recent coding agents can generate complete codebases from simple prompts, yet existing evaluations focus on issue-level bug fixing and lag behind end-to-end development. We introdu…