2 citations · 3 across the 8 of their papers we have counts for
Showing 2026Show all
3 papers · 1 filter
cs.LG2026
Geometric Self-Distillation for Reasoning Generalization
Josip Jukić, Ivan Titov
On-policy distillation provides dense teacher supervision on a language model's own trajectories. In self-distillation with privileged context, this supervision comes from the mode…
cs.LG2026
The Cancellation Hypothesis in Critic-Free RL: From Outcome Rewards to Token Credits
Tianhao Cheng, Zeyu Huang, Zihan Qiu +5
A commonly accepted explanation of critic-free RL for LLMs, based on sequence-level rewards, is that it reinforces successful rollouts with a positive advantage while penalizing fa…
cs.CL2026
A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training
Zihan Qiu, Zeyu Huang, Kaiyue Wen +16
We investigate the functional role of emergent outliers in large language models, specifically attention sinks (a few tokens that consistently receive large attention logits) and r…