Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Task Vector Descent: Learning from Non-IID Batches
Anton Baumann, Jonas Hübotter, Zeynep Akata +1
A central challenge in continual learning is to acquire new knowledge without forgetting what the model has already learned. This challenge appears in language model training when…
cs.LG2026
On Repulsive and Attractive Teachers: Separating Correctness from Behavior in Self-Distillation
Anton Baumann, Akmal Ashirmatov, Leo Schmidt-Traub +4
On-policy self-distillation provides dense, token-level supervision by conditioning a model on privileged information and distilling the resulting teacher distribution back into th…
cs.LG2026
Reinforcement Learning via Self-Distillation
Jonas Hübotter, Frederike Lübeck, Lejs Behric +8
Large language models are increasingly post-trained with reinforcement learning in verifiable domains such as code and math. Yet, current methods for reinforcement learning with ve…