10 papers
Temporal Leakage in LLM Backtesting: Measurement, Validation, and Adjusted Scores
Zeyu Zhang, Bradly C. Stadie
The standard check for contamination in LLM backtests is simple: compare scores before and after the training cutoff. We show this check is uninformative. Four flagship models fail…
All Leaks Count, Some Count More: Interpretable Temporal Contamination Detection and Mitigation in LLM Backtesting
Zeyu Zhang, Ryan Chen, Bradly C. Stadie
Backtesting LLMs on resolved events assumes models reason only from pre-cutoff knowledge, yet pretrained models inevitably leak post-cutoff knowledge. We introduce a claim-level ev…
D2 Actor Critic: Diffusion Actor Meets Distributional Critic
Lunjun Zhang, Shuo Han, Hanrui Lyu +1
We introduce D2AC, a new model-free reinforcement learning (RL) algorithm designed to train expressive diffusion policies online effectively. At its core is a policy improvement ob…
TEMPO: Temporal Enforcement via Mode-Separated Policy Optimization for Trustworthy LLM Backtesting
Zeyu Zhang, Bradly C. Stadie
Backtesting large language models on historical events requires reasoning exclusively from information available before a specified cutoff date. Yet models routinely leak post-cuto…
LAMP: Extracting Local Decision Surfaces From Large Language Models
Ryan Chen, Youngmin Ko, Zeyu Zhang +5
We introduce LAMP (Local Attribution Mapping Probe), a method that shines light onto a black-box language model's decision surface and studies how reliably a model maps its stated…
Evolutionary System Prompt Learning for Reinforcement Learning in LLMs
Lunjun Zhang, Ryan Chen, Bradly C. Stadie
Building agentic systems that can autonomously self-improve from experience is a longstanding goal of AI. Large language models (LLMs) today primarily self-improve via two mechanis…