4 papers
Correcting Stochastic Update Bias in Preconditioned Language Model Optimizers
Nikhil Nayak, Julia White, Urchade Zaratiana +7
Preconditioned optimizers are central to language model training, but their stochastic update rules are usually treated as direct approximations to population preconditioned descen…
Pioneer Agent: Continual Improvement of Small Language Models in Production
Dhruv Atreja, Julia White, Nikhil Nayak +5
Small language models are attractive for production deployment due to their low cost, fast inference, and ease of specialization. However, adapting them to a specific task remains…
Beyond Reactivity: Measuring Proactive Problem Solving in LLM Agents
Gil Pasternak, Dheeraj Rajagopal, Julia White +4
LLM-based agents are increasingly moving towards proactivity: rather than awaiting instruction, they exercise agency to anticipate user needs and solve them autonomously. However,…
ALAS: Autonomous Learning Agent for Self-Updating Language Models
Dhruv Atreja
Large language models (LLMs) often have a fixed knowledge cutoff, limiting their accuracy on emerging information. We present ALAS (Autonomous Learning Agent System), a modular pip…