23 citations · 24 across the 13 of their papers we have counts for
15 papers
Can Language Models Learn to Forecast Stock Prices
Jiacheng Guo, Suozhi Huang, Shuzhen Li +11
Post-training has been shown to significantly improve language models' performance on tasks with verifiable outcomes, including mathematical reasoning, software engineering, and co…
AhaBench: Do Agents Turn Experience into Reusable Insights? A Long-Horizon Benchmark for Continual Learning
Zerui Cheng, Jiawei Xu, Huacan Chai +3
Can language agents continually learn from experience, turning earlier interactions into reusable capabilities? AhaBench evaluates this ability through exploration after solved hid…
The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms
Jinghan Zhang, Zerui Cheng, Shiqi Chen +5
Traditional evaluations measure a learning algorithm's final performance on an i.i.d. test set, reducing learning to a single aggregate score. This approach obscures a fundamental…
TabularMath: Evaluating Computational Extrapolation in Tabular Learning via Program-Verified Synthesis
Zerui Cheng, Jiashuo Liu, Jianzhu Yao +3
Standard tabular benchmarks mainly focus on the evaluation of a model's capability to interpolate values inside a data manifold, where models good at performing local statistical s…
VeRA: Renewing Reasoning Benchmarks with Executable Specifications
Zerui Cheng, Jiashuo Liu, Chunjie Wu +5
Reasoning benchmarks need renewal along two axes: freshness and headroom. VeRA makes both executable and auditable by turning each item into a task family: a natural-language templ…
FutureX-Pro: Extending Future Prediction to High-Value Vertical Domains
Jiashuo Liu, Siyuan Chen, Zaiyuan Wang +38
Building upon FutureX, which established a live benchmark for general-purpose future prediction, this report introduces FutureX-Pro, including FutureX-Finance, FutureX-Retail, Futu…
FrontierCS: Evolving Challenges for Evolving Intelligence
Qiuyang Mang, Wenhao Chai, Zhifei Li +48
We introduce FrontierCS, a benchmark of 156 open-ended problems across diverse areas of computer science, designed and reviewed by experts, including CS PhDs and top-tier competiti…
Benchmarking is Broken -- Don't Let AI be its Own Judge
Zerui Cheng, Stella Wohnig, Ruchika Gupta +13
The meteoric rise of AI, with its rapidly expanding market capitalization, presents both transformative opportunities and critical challenges. Chief among these is the urgent need…
TAO: Tolerance-Aware Optimistic Verification for Floating-Point Neural Networks
Jianzhu Yao, Hongxu Su, Taobo Liao +4
Neural networks increasingly run on hardware outside the user's control (cloud GPUs, inference marketplaces). Yet ML-as-a-Service reveals little about what actually ran or whether…
When Hallucination Costs Millions: Benchmarking AI Agents in High-Stakes Adversarial Financial Markets
Zeshi Dai, Zimo Peng, Zerui Cheng +1
We present CAIA, a benchmark exposing a critical blind spot in AI evaluation: the inability of state-of-the-art models to operate in adversarial, high-stakes environments where mis…
AutoCode: LLMs as Problem Setters for Competitive Programming
Shang Zhou, Zihan Zheng, Kaiyuan Liu +18
Writing competitive programming problems is exacting. Authors must: set constraints, input distributions, and edge cases that rule out shortcuts; target specific algorithms (e.g.,…
LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?
Zihan Zheng, Zerui Cheng, Zeyu Shen +16
Recent reports claim that large language models (LLMs) now outperform elite humans in competitive programming. Drawing on knowledge from a group of medalists in international algor…
SPIN-Bench: How Well Do LLMs Plan Strategically and Reason Socially?
Jianzhu Yao, Kevin Wang, Ryan Hsieh +5
Reasoning and strategic behavior in social interactions is a hallmark of intelligence. This form of reasoning is significantly more sophisticated than isolated planning or reasonin…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1168
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
OML: A Primitive for Reconciling Open Access with Owner Control in AI Model Distribution
Zerui Cheng, Edoardo Contente, Ben Finch +9
The current paradigm of AI model distribution presents a fundamental dichotomy: models are either closed and API-gated, sacrificing transparency and local execution, or openly dist…