6 papers
Evolutionary System Prompt Learning for Reinforcement Learning in LLMs
Lunjun Zhang, Ryan Chen, Bradly C. Stadie
Building agentic systems that can autonomously self-improve from experience is a longstanding goal of AI. Large language models (LLMs) today primarily self-improve via two mechanis…
All Leaks Count, Some Count More: Interpretable Temporal Contamination Detection and Mitigation in LLM Backtesting
Zeyu Zhang, Ryan Chen, Bradly C. Stadie
Backtesting LLMs on resolved events assumes models reason only from pre-cutoff knowledge, yet pretrained models inevitably leak post-cutoff knowledge. We introduce a claim-level ev…
AIA Forecaster: Technical Report
Rohan Alur, Bradly C. Stadie, Daniel Kang +11
This technical report describes the AIA Forecaster, a Large Language Model (LLM)-based system for judgmental forecasting using unstructured data. The AIA Forecaster approach combin…
LAMP: Extracting Local Decision Surfaces From Large Language Models
Ryan Chen, Youngmin Ko, Zeyu Zhang +5
We introduce LAMP (Local Attribution Mapping Probe), a method that shines light onto a black-box language model's decision surface and studies how reliably a model maps its stated…
Why does in-context learning fail sometimes? Evaluating in-context learning on open and closed questions
Xiang Li, Haoran Tang, Siyu Chen +3
We measure the performance of in-context learning as a function of task novelty and difficulty for open and closed questions. For that purpose, we created a novel benchmark consist…
Learning the Market: Sentiment-Based Ensemble Trading Agents
Andrew Ye, James Xu, Vidyut Veedgav +6
We propose and study the integration of sentiment analysis and deep reinforcement learning ensemble algorithms for stock trading by evaluating strategies capable of dynamically alt…