8 papers
Spectral Souping: A Unified Framework for Online Preference Alignment
Yinlam Chow, Guy Tennenholtz, Ted Yun +4
Reinforcement Learning from Human Feedback (RLHF) effectively aligns Large Language Models (LLMs) with aggregate human preferences but often fails to address the diverse and confli…
Revisiting DAgger in the Era of LLM-Agents
Changhao Li, Rushi Qiang, Jiawei Huang +4
Long-horizon LM agents learn from multi-turn interaction, where a single early mistake can alter the subsequent state distribution and derail the whole trajectory. Existing recipes…
Exploration-Driven Optimization for Test-Time Large Language Model Reasoning
Changhao Li, Yuchen Zhuang, Chenxiao Gao +4
Post-training techniques combined with inference-time scaling significantly enhance the reasoning and alignment capabilities of large language models (LLMs). However, a fundamental…
Diffusion Controller: Framework, Algorithms and Parameterization
Tong Yang, Moonkyung Ryu, Chih-Wei Hsu +4
Controllable diffusion generation often relies on various heuristics that are seemingly disconnected without a unified understanding. We bridge this gap with Diffusion Controller (…
FlowRL: A Taxonomy and Modular Framework for Reinforcement Learning with Diffusion Policies
Chenxiao Gao, Edward Chen, Tianyi Chen +1
Thanks to their remarkable flexibility, diffusion models and flow models have emerged as promising candidates for policy representation. However, efficient reinforcement learning (…
Spectral Representation-based Reinforcement Learning
Chenxiao Gao, Haotian Sun, Na Li +2
In real-world applications with large state and action spaces, reinforcement learning (RL) typically employs function approximations to represent core components like the policies,…