11 citations · 21 across the 18 of their papers we have counts for
18 papers
Training Large Reasoning Models Efficiently via Progressive Thought Encoding
Zeliang Zhang, Xiaodong Liu, Hao Cheng +3
Large reasoning models (LRMs) excel on complex problems but face a critical barrier to efficiency: reinforcement learning (RL) training requires long rollouts for outcome-based rew…
Improving Interactive In-Context Learning from Natural Language Feedback
Martin Klissarov, Jonathan Cook, Diego Antognini +5
Adapting one's thought process based on corrective feedback is an essential ability in human learning, particularly in collaborative settings. In contrast, the current large langua…
ViSS-R1: Self-Supervised Reinforcement Video Reasoning
Bo Fang, Yuxin Song, Qiangqiang Wu +3
Complex video reasoning remains a significant challenge for Multimodal Large Language Models (MLLMs), as current R1-based methodologies often prioritize text-centric reasoning deri…
Why mask diffusion does not work
Haocheng Sun, Cynthia Xin Wen, Edward Hong Wang
The main advantages of diffusion language models over autoregressive (AR) models lie in their ability to support parallel generation and bidirectional attention, enabling a more co…
Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunities
Hao Sun, Mihaela van der Schaar
In the era of Large Language Models (LLMs), alignment has emerged as a fundamental yet challenging problem in the pursuit of more reliable, controllable, and capable machine intell…
Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback
Xiaoying Zhang, Yipeng Zhang, Hao Sun +4
Recent advances in reinforcement learning (RL) using numerical rewards have significantly enhanced the complex reasoning capabilities of large language models (LLMs). However, we i…
Rethinking Diverse Human Preference Learning through Principal Component Analysis
Feng Luo, Rui Yang, Hao Sun +5
Understanding human preferences is crucial for improving foundation models and building personalized AI systems. However, preferences are inherently diverse and complex, making it…
Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes
Katarzyna Kobalczyk, Claudio Fanconi, Hao Sun +1
As large language models (LLMs) become increasingly embedded in everyday applications, ensuring their alignment with the diverse preferences of individual users has become a critic…
The Synergy of LLMs & RL Unlocks Offline Learning of Generalizable Language-Conditioned Policies with Low-fidelity Data
Thomas Pouplin, Katarzyna Kobalczyk, Hao Sun +1
Developing autonomous agents capable of performing complex, multi-step decision-making tasks specified in natural language remains a significant challenge, particularly in realisti…
Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning
Hao Sun, Mihaela van der Schaar
Aligning Large Language Models (LLMs) is crucial for enhancing their safety and utility. However, existing methods, primarily based on preference datasets, face challenges such as…
Supervised Fine-Tuning as Inverse Reinforcement Learning
Hao Sun
The prevailing approach to aligning Large Language Models (LLMs) typically relies on human or AI feedback and assumes access to specific types of preference datasets. In our work,…
Retrieval Augmented Thought Process for Private Data Handling in Healthcare
Thomas Pouplin, Hao Sun, Samuel Holt +1
Large Language Models (LLMs) have demonstrated the strong potential to assist both clinicians and the general public with their extensive medical knowledge. However, their applicat…
Dense Reward for Free in Reinforcement Learning from Human Feedback
Alex J. Chan, Hao Sun, Samuel Holt +1
Reinforcement Learning from Human Feedback (RLHF) has been credited as the key advance that has allowed Large Language Models (LLMs) to effectively follow instructions and produce…
When is Off-Policy Evaluation (Reward Modeling) Useful in Contextual Bandits? A Data-Centric Perspective
Hao Sun, Alex J. Chan, Nabeel Seedat +2
Evaluating the value of a hypothetical target policy with only a logged dataset is important but challenging. On the one hand, it brings opportunities for safe policy improvement u…
Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examples
Hao Sun, Alihan Hüyük, Daniel Jarrett +1
Learning controllers with offline data in decision-making systems is an essential area of research due to its potential to reduce the risk of applications in real-world systems. Ho…
Reinforcement Learning in the Era of LLMs: What is Essential? What is needed? An RL Perspective on RLHF, Prompting, and Beyond
Hao Sun
Recent advancements in Large Language Models (LLMs) have garnered wide attention and led to successful products such as ChatGPT and GPT-4. Their proficiency in adhering to instruct…
Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL
Hao Sun, Alihan Hüyük, Mihaela van der Schaar
In this study, we aim to enhance the arithmetic reasoning ability of Large Language Models (LLMs) through zero-shot prompt optimization. We identify a previously overlooked objecti…
ChatHaruhi: Reviving Anime Character in Reality via Large Language Model
Cheng Li, Ziang Leng, Chenxi Yan +11
Role-playing chatbots built on large language models have drawn interest, but better techniques are needed to enable mimicking specific fictional characters. We propose an algorith…