16 papers
The State-Prediction Separation Hypothesis
Giovanni Monea, Nathan Godey, Kianté Brantley +1
Transformers use the same forward computation stream to both predict the next token and store useful state for future token predictions. We formulate the \emph{state-prediction sep…
: Better Prompt Optimization with Fewer Prompts
Zhaolin Gao, Yu, Wang +4
Prompt optimization improves language models without updating their weights by searching for a better system prompt, but its effectiveness varies widely across tasks. We study what…
Scaling Reward Modeling without Human Supervision
Jingxuan Fan, Yueying Li, Zhenting Qi +4
Learning from feedback is an instrumental process for advancing the capabilities and safety of frontier models, yet its effectiveness is often constrained by cost and scalability.…
LLMs Can Learn to Reason Via Off-Policy RL
Daniel Ritter, Owen Oertell, Bradley Guo +3
Reinforcement learning (RL) approaches for Large Language Models (LLMs) frequently use on-policy algorithms, such as PPO or GRPO. However, policy lag from distributed training arch…
Breadcrumbs Reasoning: Memory-Efficient Reasoning with Compression Beacons
Giovanni Monea, Yair Feldman, Shankar Padmanabhan +2
The scalability of large language models for long-context reasoning is severely constrained by the linear growth of their Transformer key-value cache, which incurs significant memo…
The Emergence of Complex Behavior in Large-Scale Ecological Environments
Joseph Bejjani, Chase Van Amburg, Chengrui Wang +7
We explore how physical scale and population size shape the emergence of complex behaviors in open-ended ecological environments. In our setting, agents are unsupervised and have n…