4 papers
Matching Supervision to the Student's Learning Capacity: A Unified Framework for On-Policy Self-Distillation
Yongkang Yang, Zhezheng Hao, Hong Zhang +8
On-policy self-distillation (OPSD) improves the reasoning abilities of LLMs by internalizing privileged context into model parameters through self-distillation. Two recent research…
Learned, Relied Upon, or Necessary? Separating Checkpoint Dependence from Task-Level Value in Sheaf GNNs
Yi Liu
Learned restriction maps in sheaf graph neural networks are often treated as proof that the model has discovered useful edge geometry. That conclusion does not follow from paramete…
Time is Not Compute: Scaling Laws for Wall-Clock Constrained Training on Consumer GPUs
Yi Liu
Scaling laws relate model quality to compute budget (FLOPs), but practitioners face wall-clock time constraints, not compute budgets. We study optimal model sizing under fixed time…
P-Aligner: Enabling Pre-Alignment of Language Models via Principled Instruction Synthesis
Feifan Song, Bofei Gao, Yifan Song +6
Large Language Models (LLMs) are expected to produce safe, helpful, and honest content during interaction with human users, but they frequently fail to align with such values when…