6 papers
The Flip Side of RLHF: On-Policy Feedback for Reward Model Self-Supervised Improvement
Xiaobo Wang, Tong Wu, Min Tang +3
Building strong reward models (RMs) for language model alignment is bottlenecked by the cost and difficulty of acquiring diverse and reliable preference data from human annotation…
PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectives
Zhaowei Zhang, Xiaobo Wang, Minghua Yi +5
Achieving political consensus is crucial yet challenging for the effective functioning of social governance. However, although frontier AI systems represented by large language mod…
The AI Hippocampus: How Far are We From Human Memory?
Zixia Jia, Jiaqi Li, Yipeng Kang +12
Memory plays a foundational role in augmenting the reasoning, adaptability, and contextual fidelity of modern Large Language Models and Multi-Modal LLMs. As these models transition…
Adaptive Preference Optimization with Uncertainty-aware Utility Anchor
Xiaobo Wang, Zixia Jia, Jiaqi Li +2
Offline preference optimization methods are efficient for large language models (LLMs) alignment. Direct Preference optimization (DPO)-like learning, one of the most popular approa…
ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflection
Jiaqi Li, Xinyi Dong, Yang Liu +6
We present a novel pipeline, ReflectEvo, to demonstrate that small language models (SLMs) can enhance meta introspection through reflection learning. This process iteratively gener…
In-Context Editing: Learning Knowledge from Self-Induced Distributions
Siyuan Qi, Bangcheng Yang, Kailin Jiang +5
In scenarios where language models must incorporate new information efficiently without extensive retraining, traditional fine-tuning methods are prone to overfitting, degraded gen…