2 citations · 3 across the 8 of their papers we have counts for
9 papers
Hybrid Policy Distillation for LLMs
Wenhong Zhu, Ruobing Xie, Rui Wang +1
Knowledge distillation (KD) is a powerful paradigm for compressing large language models (LLMs), whose effectiveness depends on intertwined choices of divergence direction, optimiz…
Proximal Supervised Fine-Tuning
Wenhong Zhu, Ruobing Xie, Rui Wang +3
Supervised fine-tuning (SFT) of foundation models often leads to poor generalization, where prior capabilities deteriorate after tuning on new tasks or domains. Inspired by trust-r…
Flexible Realignment of Language Models
Wenhong Zhu, Ruobing Xie, Weinan Zhang +1
Realignment becomes necessary when a language model (LM) fails to meet expected performance. We propose a flexible realignment framework that supports quantitative control of align…
Adding Alignment Control to Language Models
Wenhong Zhu, Weinan Zhang, Rui Wang
Post-training alignment has increasingly become a crucial factor in enhancing the usability of language models (LMs). However, the strength of alignment varies depending on individ…
Improving Open-Ended Text Generation via Adaptive Decoding
Wenhong Zhu, Hongkun Hao, Zhiwei He +2
Current language models decode text token by token according to probabilistic distribution, and determining the appropriate candidates for the next token is crucial to ensure gener…
Is Self-knowledge and Action Consistent or Not: Investigating Large Language Model's Personality
Yiming Ai, Zhiwei He, Ziyin Zhang +5
In this study, we delve into the validity of conventional personality questionnaires in capturing the human-like personality traits of Large Language Models (LLMs). Our objective i…
Boosting Large Language Model for Speech Synthesis: An Empirical Study
Hongkun Hao, Long Zhou, Shujie Liu +4
Large language models (LLMs) have made significant advancements in natural language processing and are concurrently extending the language ability to other modalities, such as spee…
CLEAN-EVAL: Clean Evaluation on Contaminated Large Language Models
Wenhong Zhu, Hongkun Hao, Zhiwei He +6
We are currently in an era of fierce competition among various large language models (LLMs) continuously pushing the boundaries of benchmark performance. However, genuinely assessi…
Penalty Decoding: Well Suppress the Self-Reinforcement Effect in Open-Ended Text Generation
Wenhong Zhu, Hongkun Hao, Rui Wang
The decoding algorithm is critical for open-ended text generation, transforming latent representations into coherent and meaningful outputs. This paper investigates the self-reinfo…