5 papers
Hybrid Policy Distillation for LLMs
Wenhong Zhu, Ruobing Xie, Rui Wang +1
Knowledge distillation (KD) is a powerful paradigm for compressing large language models (LLMs), whose effectiveness depends on intertwined choices of divergence direction, optimiz…
Proximal Supervised Fine-Tuning
Wenhong Zhu, Ruobing Xie, Rui Wang +3
Supervised fine-tuning (SFT) of foundation models often leads to poor generalization, where prior capabilities deteriorate after tuning on new tasks or domains. Inspired by trust-r…
Flexible Realignment of Language Models
Wenhong Zhu, Ruobing Xie, Weinan Zhang +1
Realignment becomes necessary when a language model (LM) fails to meet expected performance. We propose a flexible realignment framework that supports quantitative control of align…
Adding Alignment Control to Language Models
Wenhong Zhu, Weinan Zhang, Rui Wang
Post-training alignment has increasingly become a crucial factor in enhancing the usability of language models (LMs). However, the strength of alignment varies depending on individ…
Is Self-knowledge and Action Consistent or Not: Investigating Large Language Model's Personality
Yiming Ai, Zhiwei He, Ziyin Zhang +5
In this study, we delve into the validity of conventional personality questionnaires in capturing the human-like personality traits of Large Language Models (LLMs). Our objective i…