4 papers · 1 filter
Deeper is Not Always Better: Mitigating the Alignment Tax via Confident Layer Decoding
Xuanming Zhang, Sining Zhoubian, Yuxuan Chen +8
Autoregressive generation in large language models (LLMs) conventionally decodes from the final layer, assuming that deeper representations yield more reliable next-token predictio…
Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective
Liyuan Mao, Le Yu, Jing Zhou +7
In this work, we reveal that Large Language Models (LLMs) possess intrinsic behavioral plasticity-akin to chameleons adapting their coloration to environmental cues-that can be exp…
Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models
Binghai Wang, Yantao Liu, Yuxuan Liu +13
Generative Reward Models (GenRMs) and LLM-as-a-Judge exhibit deceptive alignment by producing correct judgments for incorrect reasons, as they are trained and evaluated to prioriti…
Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization
Xueyun Tian, Minghua Ma, Bingbing Xu +6
Supervised fine-tuning (SFT) on chain-of-thought (CoT) trajectories demonstrations is a common approach for enabling reasoning in large language models. Standard practices typicall…