4 papers
MixSD: Mixed Contextual Self-Distillation for Knowledge Injection
Jiarui Liu, Lechen Zhang, Yongjin Yang +5
Supervised fine-tuning (SFT) is widely used to inject new knowledge into language models, but it often degrades pretrained capabilities such as reasoning and general-domain perform…
Decomposing and Measuring Evaluation Awareness
Changling Li, Terry Jingchen Zhang, Jie Zhang +3
Frontier language models sometimes recognize that they are being evaluated and adjust their behavior, undermining validity of benchmark results. Yet the field studies it without a…
Fluid Reasoning Representations
Dmitrii Kharlapenko, Alessandro Stolfo, Terry Jingchen Zhang +4
Frontier large language models increasingly solve complex tasks involving abstract concepts through extended test-time thinking. Yet we lack a mechanistic account of how extended t…
The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Direction
Yihuai Hong, Dian Zhou, Meng Cao +2
Large language models (LLMs) excel on a variety of reasoning benchmarks, but previous studies suggest they sometimes struggle to generalize to unseen questions, potentially due to…