4 papers · 1 filter
Subliminal Learning as Trait-Direction Drift: A Mechanism and Targeted Control under SFT Distillation
Zhixuan Liu, Zhichen Dong, Yuyu Fan +2
Beyond intended capabilities, model distillation can transfer hidden traits from a teacher. A teacher biased by a system prompt can generate semantically clean training data, such…
Scaling Model-Generated Distillation Data Can Make Latent Teacher Traits More Recoverable
Zhichen Dong, Zhixuan Liu, Yuyu Fan +3
Scaling model-generated data is usually viewed as improving distillation: more examples should increase coverage, reduce noise, and produce stronger students. We show a second effe…
Native Reasoning Models: Training Language Models to Reason on Unverifiable Data
Yuanfu Wang, Zhixuan Liu, Xiangtian Li +2
The prevailing paradigm for training large reasoning models--combining Supervised Fine-Tuning (SFT) with Reinforcement Learning with Verifiable Rewards (RLVR)--is fundamentally con…
Adversarial Preference Learning for Robust LLM Alignment
Yuanfu Wang, Pengyu Wang, Chenyang Xi +13
Modern language models often rely on Reinforcement Learning from Human Feedback (RLHF) to encourage safe behaviors. However, they remain vulnerable to adversarial attacks due to th…