6 papers
Understanding the Challenges in Iterative Generative Optimization with LLMs
Allen Nie, Xavier Daull, Zhiyi Kuang +10
Generative optimization uses large language models (LLMs) to iteratively improve artifacts (such as code, workflows or prompts) using execution feedback. It is a promising approach…
Trust the Batch, On- or Off-Policy: Adaptive Policy Optimization for RL Post-Training
Rasool Fakoor, Murdock Aubry, Nicholas Stranges +1
Reinforcement learning is structurally harder than supervised learning because the policy changes the data distribution it learns from. The resulting fragility is especially visibl…
Offline Learning and Forgetting for Reasoning with Large Language Models
Tianwei Ni, Allen Nie, Sapana Chaudhary +3
Leveraging inference-time search in large language models has proven effective in further enhancing a trained model's capability to solve complex mathematical and reasoning problem…
AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents
Ke Yang, Yao Liu, Sapana Chaudhary +4
Autonomy via agents using large language models (LLMs) for personalized, standardized tasks boosts human efficiency. Automating web tasks (like booking hotels within a budget) is i…
From Demonstrations to Rewards: Alignment Without Explicit Human Preferences
Siliang Zeng, Yao Liu, Huzefa Rangwala +3
One of the challenges of aligning large models with human preferences lies in both the data requirements and the technical complexities of current approaches. Predominant methods,…
Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokens
Zhepeng Cen, Yao Liu, Siliang Zeng +4
Language models are often trained to maximize the likelihood of the next token given past tokens in the training dataset. However, during inference time, they are utilized differen…