5 papers
Towards Scalable Pre-training of Visual Tokenizers for Generation
Jingfeng Yao, Yuda Song, Yucong Zhou +1
The quality of the latent space in visual tokenizers (e.g., VAEs) is crucial for modern generative models. However, the standard reconstruction-based training paradigm produces a l…
Expanding the Capabilities of Reinforcement Learning via Text Feedback
Yuda Song, Lili Chen, Fahim Tajwar +5
The success of RL for LLM post-training stems from an unreasonably uninformative source: a single bit of information per rollout as binary reward or preference label. At the other…
Maximum Likelihood Reinforcement Learning
Fahim Tajwar, Guanning Zeng, Yueer Zhou +7
Reinforcement learning (RL) is the method of choice for training models in setups where the objective function can only be evaluated by sampling from the model. Our key observation…
Outcome-based Exploration for LLM Reasoning
Yuda Song, Julia Kempe, Remi Munos
Reinforcement learning (RL) has emerged as a powerful method for improving the reasoning abilities of large language models (LLMs). Outcome-based RL, which rewards policies solely…
Accelerating Unbiased LLM Evaluation via Synthetic Feedback
Zhaoyi Zhou, Yuda Song, Andrea Zanette
When developing new large language models (LLMs), a key step is evaluating their final performance, often by computing the win-rate against a reference model based on external feed…